Written by Per Gulløv Lundh Eeg (Head of AI & Digitalisation og Partner i LEAD+) and Simon Stubben (Ekspert Rådgiver & Ejer af TIWAS TECH)
AI is on everyone’s lips, and many companies are eager to jump on the bandwagon. But before you dive into implementing AI in your organisation, it’s important to understand that AI is far from “plug-and-play.” While it’s easy to experiment with external AI services (i.e., those you access online via your browser, such as Chrome or Edge) like ChatGPT or Gemini, the picture becomes far more complex when you want an AI solution that can accurately answer your business-specific questions—while also complying with rules and standards.
A context-specific language model that needs to handle your company’s data requires solid data governance. If you don’t have control over what is personally sensitive, business confidential, or subject to legislation, you risk leaking information that can damage both your reputation and compliance. So, how do you avoid the pitfalls when getting started with AI? Here’s a guide on how to do it right.
Bring the large language model into your small organisation
Leverage an existing large language model, such as Mistral AI (France), ChatGPT (USA), or Deepseek (China), which already has the foundational training in place. By moving it into your own secure environment, you can:
- Handle personal and business-sensitive data securely
- Reduce hallucinations
- Create dedicated solutions with access to your organisation’s own data
Without having to train a large language model from scratch.
And it’s secure. Not because France, the USA, and China are secure countries (France is probably still quite secure for Danish data), but because you take a language model and move it into your own secure IT environment.
A strong foundation is crucial
Many companies quickly discover that it’s not just about “getting an AI.” Often, it’s the strategic approach to AI and control over data that are lacking. Before a company-specific AI can make sense, it’s essential to have a good handle on your IT landscape, your data streams, and the quality of the data you’ll feed your language model. At the same time, you need to incorporate requirements from GDPR and the EU’s AI Act into your strategy.
ChatGPT or Gemini can certainly be used for general purposes, but an AI that needs to handle specific business data should live in an environment where the data’s lifecycle is controlled from collection to deletion. This requires technical and organisational maturity, especially if you have a more complex setup with many services and data sources.
Integrated into the architecture – not in silos
To optimally utilise an AI solution, it needs access to all relevant parts of the company’s data and context—not just run in isolation in a smaller part of the business. It’s about integrating the AI solution into your IT architecture and data landscape. For example, an IT landscape built on a service- or event-driven architecture can continuously feed language models with data from various systems. This way, you can prevent your AI solution from reinforcing silos.
On the way to an AI-driven future
AI can be a significant advantage for everything from internal Q&A and document management to automated customer support. But typical mistakes occur when data is not in order, when the AI hasn’t been thoroughly tested in a pilot phase, or when employees aren’t properly trained in using an AI tool. It’s about allocating time and resources to establish a solid knowledge base among both leaders and employees, setting up a data governance model, establishing a permission framework to set boundaries, and monitoring performance with KPIs that measure efficiency, quality, and ROI.
To prevent employees from experimenting with external AI tools and risking the sharing of confidential data, you should offer a company-specific AI model. It can be used like any other public AI model to start with, and gradually include data for internal knowledge search, document management, or customer support. This method ensures a controlled and compliant AI solution for your company.
If you get off to a good start, AI can become an integrated part of your business processes. In the long run, you can shut down external services that pose a security risk. It’s also crucial to stay updated on legislation, technological possibilities, and best practices. By combining compliance and innovation, you can reap the benefits AI can offer—without taking unnecessary risks or losing control of your data.
What to remember when getting started with AI:
- Build an AI strategy: What does your organisation want to achieve with AI?
- Get control of your data: A solid data strategy and data governance architecture are essential. Without them, you risk building on shaky ground.
- Think compliance from the start: Standards like ISO 27701 (privacy protection) and ISO 42001 (AI implementation) can help meet requirements and regulations from, for example, GDPR, ISO standards, and the EU’s AI Act, and should be an integral part of your AI strategy.
- Integrate AI into your existing IT architecture: Avoid silos and ensure that the AI has access to the right context.
- Test thoroughly in a pilot phase: Before scaling, make sure the AI works as it should.
- Offer an internal AI model: This way, you prevent employees from using external tools that could pose a security risk.
The future of AI in companies
If companies get a handle on the fundamental pitfalls, AI can become a natural and highly significant part of standard processes. We will increasingly see AI agents being used for support, process optimisation, and service operation. And the companies that are most effective and get the most out of their AI model have thought AI into their strategy, trained their employees, and trained their models on their own data, thereby ensuring they are context-specific.
AI is a gamechanger for companies, but only if implemented correctly. By avoiding the pitfalls and setting a solid strategy, you can ensure that your organisation is ready to reap the benefits—without losing control.
Leave a Reply