Hamza from Security Dimension recently ran an experiment to determine if ChatGPT is up to the challenge of tackling security vulnerabilities in code. He tried to get solve a secure programming challenge in Cross Site Scripting in both Python and JavaScript. His observations were that ChatGPT was particularly good at handling Python code but struggled in JavaScript. It constantly struggled with usability requirements, and unless strictly guided would end up making a lot of breaking changes to the existing code. With enough information and context, it was able to create a good enough solution to pass the challenges but the solutions did not address the root of the security vulnerability. Read more about Hamza’s secure coding ChatGPT experience on Security Dimension.
This segment was created for the It’s 5:05 podcast