Would you want to insure people who think they have no responsibility because they've delegated it to an AI? They might as well have delegated the responsibility to a child or a dog. To sell them insurance, you as the insurer are making a financial bet on the ability of the dog to take care of anything that does go wrong.
And still as the insured, using the AI imbued with your responsibility risks horrible outcomes that could still ruin your life. The AI has no life to ruin. It was never really responsible.
It's just a numbers game. Set your premiums such that you take in more than you pay out. If losses due to dumb use of AI are common then the premiums will be high, but there's no reason to refuse to issue such policies altogether.
Would you want to insure people who think they have no responsibility because they've delegated it to an AI? They might as well have delegated the responsibility to a child or a dog. To sell them insurance, you as the insurer are making a financial bet on the ability of the dog to take care of anything that does go wrong.
And still as the insured, using the AI imbued with your responsibility risks horrible outcomes that could still ruin your life. The AI has no life to ruin. It was never really responsible.