Revisiting AI risk: 3 more threats shaping the future

AI hallucinations occur when a model generates information that sounds convincing but is actually false or misleading. These errors often arise when an AI system attempts to fill in gaps in its training data with incorrect assumptions or patterns. In high-stake environments like healthcare, law, finance or engineering, such hallucinations can easily go unnoticed and be relied upon, such as how two US lawyers cited non-existent legal cases in a court filing after using ChatGPT.

When AI generated misinformation is treated as fact, the fallout can be severe, leading to misdiagnoses, flawed legal advice, financial errors and structural damage. That makes strong verification processes, ongoing human oversight and comprehensive insurance to support AI integration and usage more vital than ever. 

The black box dilemma is the challenge of understanding how and why an AI system arrives at a particular decision. While we can often see the inputs and final outputs, the inner workings of complex models remain largely opaque. This can create a major hurdle when it comes to building trust, ensuring accountability and assessing whether an AI is acting fairly or without bias.

Imagine an autonomous vehicle faced with a split-second decision at a yellow light. It chooses to accelerate instead of stopping and causes a minor collision. If no one can explain how or why the AI reached that decision, it becomes challenging to assign responsibility or correct the error. Ultimately, a continued lack of transparency around how AI systems reach their conclusions can erode trust. As companies continue to explore new ways to use these tools, comprehensive insurance can offer a valuable safety net—helping support AI adoption with greater confidence.

AI-generated content is creating new challenges in the world of IP, raising questions around originality and ownership. Generative AI models are often trained on vast datasets that can contain copyrighted, trademarked or patented material, raising the risk that outputs could unintentionally replicate protected material. Without careful oversight, businesses using AI may find themselves falling foul of the rules by infringing on existing IP.

Take a look at Getty Images Inc and others v Stability AI Ltd. In this case, Getty Images alleged that Stability AI trained its Stable Diffusion model on copyrighted images without its permission and enabled a range of AI-generated outputs that allegedly infringed on their IP rights. This illustrates how AI models can lead to unauthorized reproduction which raises significant legal and ethical concerns around IP. With the legal framework still catching up, companies adopting AI for content creation or distribution should tread carefully and consider insurance to safeguard against potential financial losses and reputational fallout.