Could you use AI to come up with an unsolvable crime?

There are no guard rails, or there won’t be for long. That said, AI can also solve said crime. No crimes are unsolvable.

Yes. Under certain circumstances.

You can ASK.

It can generate a response, which like all its responses, will be based on pseudo-random choices of words other people wrote, correlated in ways no one really understands, to create a re-mix that may or may not sound more or less like a weird vaguely insane person with a surprising amount of willingness to gush forth about, more or less, the subject you input.

The odds of it actually generating an actually “unsolvable crime”, seem vanishingly low to me, unless perhaps you multiply it by the near-infinite times you can ask it (but there is a very amount of time people would be willing to spend reading the answers).

Those types of AI apps are NOT actually engaging the problem. They are just picking a mix of words to list, based on statistical associations with the words in your input, of a data set based on trawling a huge number of Internet texts.

“Is there anything inherent in AI structure that would act as a guard rail against criminality?” – LOL NO, though some such AI’s do attempt to filter out some things, but since it doesn’t understand any actual concepts, let alone the law, it can’t really do that. It can only reduce the frequency, based on some filters and/or training (where again, training is about data associations, not understandings).

“Is AI smart enough to plan the perfect crime?”
– No. And AI itself is not “smart”.

A human might be able to plan some crimes that might be hard to solve with the aid of AI tools, though.