Has anyone thought to try using GPT4o to observe, direct, or hypothesize on lab experiments?…

With the new multimodal and rapid interactions capable via a smartphone, it begs the question of what could you use it for beyond simple conversations. Without jumping straight to building skynet, it would be an interesting experiment to let GPT4o interact with real world lab experiments via observation and communication via a smartphone. Setup a tripod live stream the video of your lab table experiment, interact with the model live during the experiment via chat, and just see how it goes.

What might it be able to deduce, help out with, document, or otherwise contribute? These are all the steps to getting to real world “copilots” with GenAI. I wonder what you could do with it?