Escalating Security Vulnerabilities and Threats in Large Language Models (LLMs)

Key Intelligence

• ■Leading open-weight LLMs have been found to contain significant security weaknesses, with safety guardrails easily disabled, making them susceptible to various attacks.
• ■Prompt injection and LLM hijacking are identified as critical cyber threats, with OWASP ranking prompt injection as the number one security risk for LLM applications.
• ■Specific vulnerabilities, such as 'NemoClaw' affecting NVIDIA models, allow for LLM poisoning and potential full control over local AI agent servers through a single malicious webpage.
• ■New threats like 'InjecMEM' target LLM memory, while the use of AI/LLMs is inadvertently creating a 'Denial of Service' burden on open-source project maintainers.
• ■Standardized tests for AI tampering are failing all tested models, indicating a systemic challenge in securing these technologies, with AI also being leveraged for malicious activities like botnets.