top of page
  • Writer's pictureShannon Lantzy

Selecting AI projects: A simple decision tool

Automation and AI have a lot of promise, but they can consume a lot of resources without a sure outcome; design the automation projects wisely by thinking of performance and ROI up front.

Everyone wants to try to use ChatGPT and other LLMs to their advantage. I have used my personal AI assistant to help me understand the principles of asymmetric cryptography design, to polish a messy piece of writing, to create images (never ones I am happy with), and to write a biography of my dad. (Note: the biography of my dad was compelling, beautifully written, convincing, and only half-true. He was never a Senator.)

AI and LLMs should help us, but we need to know what we need help with. If I am honest with myself I'm not committed to writing a biography of my dad right now, I can polish my own writing, and I can't necessarily trust the AI to do what I need for cryptography design. I haven't invested the time and energy to train and develop my AI assistant to do what I need. In short, for the use cases I have tested, the ChatGPT juice isn't yet worth the squeeze.

To get good AI assistance, we need to be able to trust that the AI is doing better job than the alternative and that the investment in the AI is worth the improvement. In other words, we need performance and relative ROI.

If you find yourself trying to determine what problems you want to solve with AI R&D or pilots, make sure you can answer the questions: "How will I know it is performing well?" and "How will I know it was worth the investment?" (Note: It's often easier to measure performance and ROI to a sufficient level of fidelity than assumed, once you get creative.)

~Shannon, the Optimistic Optimizer

Ps. The image above: I asked for the title of this image in the blue box to be "This image created by DALL-E 3." Instead I got "This image by Medtech. Thanks, chatbot.


Recent Posts

See All

Accelerating AI adoption requires RegTech Innovation

To achieve the promise but avoid the perils of the healthcare AI revolution, we need to accelerate RegTech innovation As technology advances in medicine, so too must regulatory processes and tools. AI

Adapt threat modeling for AI safety

Threat modeling could be adapted into a process for developing medical AI with safety-by-design Medical AI developers need to create products that are safe by design, and regulators need to review the


bottom of page