In general, most modern vulnerabilities are initially identified with fuzzing systems under abnormal conditions. Whether these issues may be consistently exploited can be probabilistic in nature, and thus repeatability with a POC dataset is already difficult.
That being said, most modern exploits are already auto-generated though brute-force, as nothing more complex is required.
>Does anyone disagree?
CVE agents already pose a serious threat vector in and of itself.
1. Models can't currently be made inherently trustworthy, and the people claiming otherwise are selling something.
"Sleeper Agents in Large Language Models - Computerphile"
2. LLMs can negatively impact logical function in human users. However, people feel 20% more productive, and that makes their contributed work dangerous.
3. People are already bad at reconciling their instincts and rational evaluation. Adding additional logical impairments is not wise:
4. Auto merging vulnerabilities into opensource is already a concern, as it falls into the ambiguous "Malicious sabotage" or "Incompetent noob" classifications. How do we know someone or some models intent? We can't, and thus the code base could turn into an incoherent mess for human readers.
Mitigating risk:
i. Offline agents should only have read-access to advise on identified problem patterns.
ii. Code should never be cut-and-pasted, but rather evaluated for its meaning.
iii. Assume a system is already compromised, and consider how to handle the situation. In this line of reasoning, the policy choices should become clear.
That being said, most modern exploits are already auto-generated though brute-force, as nothing more complex is required.
>Does anyone disagree?
CVE agents already pose a serious threat vector in and of itself.
1. Models can't currently be made inherently trustworthy, and the people claiming otherwise are selling something.
"Sleeper Agents in Large Language Models - Computerphile"
https://www.youtube.com/watch?v=wL22URoMZjo
2. LLMs can negatively impact logical function in human users. However, people feel 20% more productive, and that makes their contributed work dangerous.
3. People are already bad at reconciling their instincts and rational evaluation. Adding additional logical impairments is not wise:
https://www.youtube.com/watch?v=-Pc3IuVNuO0
4. Auto merging vulnerabilities into opensource is already a concern, as it falls into the ambiguous "Malicious sabotage" or "Incompetent noob" classifications. How do we know someone or some models intent? We can't, and thus the code base could turn into an incoherent mess for human readers.
Mitigating risk:
i. Offline agents should only have read-access to advise on identified problem patterns.
ii. Code should never be cut-and-pasted, but rather evaluated for its meaning.
iii. Assume a system is already compromised, and consider how to handle the situation. In this line of reasoning, the policy choices should become clear.
Best of luck, =3