Six top tips for evaluating and selecting COTS vendors and solutions

In Axiologik’s recently published technology report ‘Automotive Finance and Mobility Solutions – the New International Landscape’ we highlighted how increasing sector convergence, the rise of multimodal mobility, subscription and MaaS, along with developments in robotics, AI and wider technologies, are serving to further complicate an already crowded and confusing software ecosystem.

Solutions from both new fintech entrants and existing sector vendors have proliferated over the last few years, with COTS vendors increasingly presenting bold sales pitches and slick product demonstrations promising microservices powered modularity, end-to-end lifecycle functional richness, intrinsic omnichannel capabilities and flexible, configurable product support (loan, lease, rental, mobility).

This has led to a situation in which selecting vendors and platforms is highly complex and often difficult to successfully achieve without an encompassing evaluation framework and access to specialist, external decision support assistance. Making the wrong choice is a significant risk when entering into costly, long-term technology partnership arrangements.

In this short article we highlight six key subject areas to explore and investigate with vendors as part of a comprehensive evaluation and selection framework. These are areas of investigation that are often missed or shied away from during evaluation and selection but which should be explored openly with the vendor in order to establish a true understanding of their solution and wider implementation, service transition and service management capabilities.

Key Subject Areas:

1. How will the solution land?

The biggest failing any organisation makes in the implementation of new technology is thinking the software alone will transform their business. This is rarely, if ever the case.

Typically, for new solutions to be successful requires operating model changes, allied to new processes and procedures. Existing staff not only need to be trained in the software but also in the new ways of working within the organisation. It’s an old mantra, but think “People, Process, Technology”.

It is therefore critical at the outset to consider how the solution will land. What impact will it have on the business? How does the business need to re-shape to fully realise the value of the investment?

These are things that will not be considered by the vendor. It is therefore necessary to establish a “client-side” team to manage the organisational change and provide an interface with the vendor. This team should be established as early as possible in the process, ideally to run the selection process such that is has full ownership of the success outcomes from the outset.

2. Product and service coverage – what is the real-world capability?

COTS platform vendors often claim their solutions are fully flexible, encompassing functionality to support loan (e.g. HP, PCP), leasing (business and personal), rental (short and longer term) and emerging mobility and usage product use cases (e.g. flexible subscription, car-sharing, car-pooling etc). In reality, these bold assertions need to be investigated and explored with the vendor to establish if the solution truly possesses this real world ‘hybrid’ product support capability.

Simply being able to configure a loan or leasing product is very different to being able to support the complex downstream processes. Also, can the vendor really evidence that they have supported this product and operating model complexity with a major automotive enterprise?

Truly flexible, highly modular, hybrid platforms that support a diverse asset portfolio and complex, wide-ranging business and operating models are extremely rare in the international market - many COTS vendors that have historically focused on supporting the retail finance sub-sector are struggling to re-architect and re-engineer their solutions to support this new world of convergence, subscription and pay-on use mobility. This capability needs to be carefully explored with the vendor using detailed evaluation scenarios and use cases, covering the full end to end lifecycle, which should be reviewed in carefully structured ‘deep-dive’ solution review sessions.

3. Will I work with the vendor 'A Team'?

Software vendors rarely identify the project team members that will help implement their component or platform solution, until very late in in the decisioning process. It's often not until automotive companies have confirmed their selection decision and implementation planning has commenced that the vendor may introduce you to the project team delivering the solution - this is much too late!

The difference between a software vendor's project 'A’ team and its 'B’ or ‘C’ teams can be substantial in terms of capability and knowledge and can significantly impact the speed of delivery and the quality of the end implementation.

Automotive companies need to establish the skills, experience, and track record of the vendor's proposed team up-front as part of the selection process:

Are they experienced with the product?

Have they implemented the solution before in a similar environment? 

Will this be their first project in the automotive sector and thus they may lack an understanding of sector operating models, processes, and requirements?

Does the vendor have implementation capacity and lead time challenges which means that the ‘A team’ resources are already ring-fenced by existing clients?

It pays to be direct and open with your prospective vendor. Ensure you are dealing with the vendor's 'A team' and not an inexperienced team learning about the software product and the automotive industry on your implementation project!

It is entirely reasonable to ask the vendor to introduce the project team, their experience and proposed role / allocation model to your project. Ask for the names of the proposed project team and review their LinkedIn profiles, probing deeply into their individual experience and capabilities – this is entirely acceptable and logical and serves to demonstrate that you will not accept a sub-optimal organisation and resource model for your project. 

4. What is the implementation starting point?

Over the last few years, many auto finance software suppliers have recognised the significant benefits in delivering their solutions with an 'out of the box' reference set-up or pre-configuration as a starting point for client projects. This is a potentially valuable 'accelerant' within their overall implementation methodology.

Reference configurations will typically include elements such as local taxation, regulatory and fiscal rules, product and service constructs, payment profiles, organisational role types, process flows, and standard industry workflows. Software suppliers also include within pre-configuration standard 'pre-plumbed' interfaces developed on previous projects across the broader ecosystem. Increasingly vendors also bring standard OTB digital portal and App toolsets (e.g. Driver or Fleet Manager self-serve capabilities) that can be used as a starting point and foundational architectural framework from which clients can develop, richer bespoke digital and omnichannel solutions. 

The reality of reference set-ups is that they are broad and highly variable in their level of detail, quality, and re-usability. Many vendors use the availability of reference configurations as a key tenet of their sales pitch but the reality of reusability, acceleration and wider implementation benefits in the real world is often disappointing.

Automotive companies need to investigate what the vendor is providing on day one as the starting point for implementation and the benefits this will bring to your implementation, consider:

Is this reference configuration relevant and applicable to your specific target operating model and implementation scenario?

What are the potential real-world re-use opportunities on your implementation project?

What is detailed specification of the starting configuration – e.g. Products? Services? Channel coverage? Interfaces / Integration APIs? Workflows? etc

Will this help accelerate delivery timescales on your project, or is this just a 'smoke and mirrors' capability statement from the vendor with no real-world substance?

5. End User Configuration – a myth or reality?

A key theme that has developed over the last decade is the need for software providers to provide intrinsic flexibility, including comprehensive end-user parameterisation, configuration and customisation capabilities, within their software solutions.

Flexibility and agility are broad concepts, so it pays to question and test the usability of a system and its features which help to facilitate rapid organisational change, innovation and autonomy. Consider if the system can allow appropriately trained internal users to rapidly set up and modify vital product elements, pricing, workflow, and even user interface design through 'end user' configuration facilities without the supplier's intervention or complex code development?

End-user configuration capabilities promote 'self-sufficiency', limiting the client's need for additional software development and customisation by the software vendor. This autonomy also brings Total Cost of Ownership (TCO) benefits through reduced post-implementation vendor development costs and reductions in overall software change pipelines and backlogs.

The reality of end-user configuration is that nearly every vendor will claim to provide it within their solution or 'have it on our product roadmap', but often the configuration capability provided varies significantly. It is thus critical that automotive companies get 'under the bonnet' and establish the true extent and sophistication of end-user configuration, consider:

What skillsets/training might be required?

What is the complexity?

Are there specific tools and templates within the product to support configuration?

What degree of adoption and ongoing BAU model does the vendor have with previous clients?

Are there demonstrable case studies and real-world benefit metrics available?

6. Don’t forget about Service

In our experience evaluation and selection projects often fail to focus on the end-to-end value stream, thus ignoring a vendor’s service transition and service operations capabilities, which are critical elements of a long-term strategic relationship and ultimate partnership success.

Often the evaluation focus is heavily weighted to product capabilities, architecture and implementation with service evaluation criteria being light and limited to relatively generic SLA, help-desk, and release management reviews.

Automotive companies really need to drill down to the next level of evaluation detail with prospective vendors considering such key subjects as scalability, reliability, security, and service operations – key areas of exploration and supporting questions should include:

Scalability/ Reliability

How easy it is to scale the platform up and down during peak/ quiet times? Is it an automated flexible scale model based on thresh holds or manual?

How does the scaling affect the cost profile? How is this triggered?

How is capacity managed including the use of business forecasts, models and testing against those?

What load / scale testing do you do and does this take account of cyclic demand?

Security

Ensure that the vendor explains the security responsibility model? Where do my accountabilities start and stop as a client compared to the vendor?

How is the Joiners/ Movers/ Leavers process triggered to ensure security compliance?

Does the vendor maintain a software bill of materials and understand the dependencies with all software license and open-source software?

How does the vendor handle and manage security for third-party services and vendors?

Service Operations

Ensure the vendor explains (and has documented) the process and timescales to get normal, standard, and emergency changes into production safely?

What level of observability of your platform/ solution does the vendor implement and how is this used to improve performance/ services/ reliability/ scalability?

What are you using to automate your build, test, and deployment pipelines?

What is the means of prioritising the backlog including non-functional, technical debt? Does this include the client, product manager?

Summary – Axiologik Evaluation, Selection & Decision Support Services

At Axiologik, we work with OEMs, finance, leasing, and mobility enterprises. We support clients with evaluation, decision-making and due diligence guidance when replacing increasingly costly, restrictive, legacy solutions or supporting new ‘greenfield’ channel and product launches.

Axiologik consultants are decision support specialists assisting automotive organisations with evaluation & selection projects across technology, application and BPO decisioning. We bring a deep knowledge of the international automotive COTS landscape (platforms & components) including unique vendor and product insights & intelligence.

Independent, product agnostic and highly experienced, we bring out of the box methodologies, frameworks and tools to accelerate and catalyse evaluation projects – this includes comprehensive ‘best practice’ automotive finance, leasing and mobility business requirement catalogues and evaluation criteria covering the full scope of vendor and solution evaluation – e.g. architecture & technology, connectivity, modularity & deployment options, data management, analytics & BI, vendor implementation capability & track record, service transition, service operations etc

Our decision support services cover the entire procurement process, from requirements gathering to short-listing and recommending the right solution and vendor for your business. Importantly this includes proven 3rd party commercial negotiation and contractual support services which help automotive companies optimise and de-risk typically long-term strategic technology relationships.

Our capabilities are evidenced through multiple successful automotive sector case studies.

Like most in-depth market reports, the recently published ‘Automotive Finance & Mobility Solutions – the new International Landscape’ report tells only part of the story. We are therefore happy to share the 'Directors Cut' voiceover, including more revealing insight, take-aways, views and market intelligence regarding specific COTS vendors, in a “private” one-to-one or group session.

Other clients have found this approach extremely valuable as they consider future IT architecture design and application selection and need to get an objective, real-world viewpoint on vendor and solution capabilities.