You are complaining that developers can't keep up with vibe coded features. The solution might actually be more AI.
There's an opportunity to automate some of the QA traditionally done manually. I tried this last week on our main app (not some toy thing):
(turn on agent mode in chat gpt)
"Hey put on your QA hat and test <url> with <user> and password. Give me detailed feedback on UX, bugs, etc."
I was being lazy here with my prompting. But it works shockingly well on anything browser based. It will start using whatever you point it at and do things that normal users might do. Obviously, you can give it more detailed guidelines on what to test, what to ensure is working, etc.
I got a pretty detailed report back and most of it was valid/constructive. I'm planning to do more of this. It beats developers doing QA (they are too biased usually and seem reluctant to do it) and we don't have any dedicated QA people. Manual QA can be very expensive. I don't think the need for that totally goes away, but you should probably focus that on the most valuable/hardest things to test.
In any case, it's pretty cool to watch Chat GPT explore a UI and attempting to use features. It's very thorough and it seems to figure out workarounds for UX issues as well when things don't work as expected. This is exactly the kind of stuff that developers testing their own UIs are blind to. They know where to click and don't even think about it.
A related issue is actually updating documentation and marketing material with up-to date screen captures and screenshots. Annoyingly, Chat GPT doesn't allow me to save the videos it takes of the AI using the browser. But that stuff could actually be documentation gold. Doing this manually is very tedious.
AI is very good for mirroring the culture around it. You can use it for all kinds of things, including QA, but you still need a culture where QA and UX matter.
MS doesn't have that. It's a B2B marketing company with a rapidly decreasing developer headcount glued to the back, far away from daylight.
There's an opportunity to automate some of the QA traditionally done manually. I tried this last week on our main app (not some toy thing):
(turn on agent mode in chat gpt)
I was being lazy here with my prompting. But it works shockingly well on anything browser based. It will start using whatever you point it at and do things that normal users might do. Obviously, you can give it more detailed guidelines on what to test, what to ensure is working, etc.I got a pretty detailed report back and most of it was valid/constructive. I'm planning to do more of this. It beats developers doing QA (they are too biased usually and seem reluctant to do it) and we don't have any dedicated QA people. Manual QA can be very expensive. I don't think the need for that totally goes away, but you should probably focus that on the most valuable/hardest things to test.
In any case, it's pretty cool to watch Chat GPT explore a UI and attempting to use features. It's very thorough and it seems to figure out workarounds for UX issues as well when things don't work as expected. This is exactly the kind of stuff that developers testing their own UIs are blind to. They know where to click and don't even think about it.
A related issue is actually updating documentation and marketing material with up-to date screen captures and screenshots. Annoyingly, Chat GPT doesn't allow me to save the videos it takes of the AI using the browser. But that stuff could actually be documentation gold. Doing this manually is very tedious.