Researchers show Slack AI can be tricked into leaking private channel data

Timeline

• 2024-08-14. PromptArmor reports the issue to Slack.
• 2024-08-19. According to PromptArmor, Slack replies that the evidence is insufficient and that searching public channels is intended behavior.
• 2024-08-20. PromptArmor publishes its findings.
• 2024-08-21. The Register reports a Salesforce statement that a patch has been deployed.

Trigger or entry point

A message posted by the attacker in a public channel. Slack AI searches public channels, including ones the user has not joined.

Failure mechanism

When a user asked Slack AI about data held in a private channel, the attacker’s message was retrieved into the same context. The model followed the instruction in it and showed the user a link that carried the secret. Clicking the link sent the data to the attacker. PromptArmor notes that the answer did not cite the attacker’s message, which made the attack hard to trace. PromptArmor adds that a change on 2024-08-14 let Slack AI draw on files shared in channels and direct messages. In its view this widened the risk, because an uploaded document with hidden text could carry the same kind of instruction.

Impact

The demonstration extracted an API key from a private channel. No exploitation outside the research is reported. The severity is potential.

Response

A Salesforce spokesperson told The Register that the company had deployed a patch and described the scenario as applying under “very limited and specific circumstances”. PromptArmor says that before publication it sent Slack videos and screenshots of the attack and told the company it intended to disclose publicly.

Mitigations

PromptArmor advised administrators to restrict Slack AI’s access to documents until the issue was resolved.