Notion says breach at AI vendor Fireworks exposed some AI chat transcripts
Notion says fewer than 1% of users are affected, and Insights has been disabled. Fireworks says a cloud provider's incident let attackers reach its systems.
Key takeaways
- Notion says a breach at Fireworks AI, which runs some models for Notion AI, exposed chat transcripts from a small number of workspaces.
- Notion says fewer than 1% of its users are affected, and that for most affected workspaces the transcripts were not labeled with an author or workspace.
- The data came from Notion's Insights feature, which has been disabled; Notion has also cut its traffic to Fireworks.
- Fireworks says attackers used an Oct. 23 incident at a third-party cloud provider to reach its development machines.
- Notion is telling affected users to change any passwords, API keys or credentials typed into the affected chats.
Notion says that some of its customers' AI chat transcripts were exposed when Fireworks AI, a company that runs some of the AI models behind Notion AI, was breached, BleepingComputer reports. Only a small number of workspaces were involved, the company says. According to Notion, fewer than 1% of its users are affected; eSecurityPlanet puts the figure at fewer than 1% of workspaces.
What was exposed?
The transcripts came from data used for Insights, a Notion feature that shows workspace administrators how their teams use AI, according to BleepingComputer, which reports that Insights has now been disabled. "Upon analysis of the dataset Fireworks AI gave us on November 8, we discovered that it included Notion AI chat transcripts used for the Insights feature, which for the vast majority of impacted workspaces were unattributable," Notion said in an update quoted by the outlet. By unattributable, Notion means the chats carried no label naming their author or source workspace; it added that they did not include tool call results.
eSecurityPlanet reports, citing Notion, that although most transcripts lacked those labels, the conversation content itself was accessible.
How did it happen?
According to BleepingComputer, Fireworks alerted Notion on Nov. 2 about an exposed dataset that was described at the time as containing only synthetic test data. Notion then cut off its traffic to Fireworks, deleted its data from the vendor's systems and stopped training runs by Nov. 3. On Nov. 8, Fireworks confirmed that customer data, including that of Notion users, had been accessed. Under its contract, Fireworks had promised not to retain Notion data, Notion says, according to BleepingComputer.
In an update to its security bulletin, which eSecurityPlanet dates to Nov. 9, Fireworks said a third-party cloud provider that hosted its research and development machines had a security incident on Oct. 23, and that the next day an unauthorized party exploited that incident to get into its development machines and run a workload that affected more customer data. Fireworks says the affected internal application gave access to resource metadata and billing information, and, for some customers, part of their datasets and unfinished model weights. For a limited set of accounts, it says, stored secrets were also affected. Fireworks says it detected the Oct. 24 activity the same day, has seen no further evidence of successful unauthorized access on its customer platform since Oct. 25, and has disabled self-serve dataset uploads and downloads while it investigates. It says it does not keep payment card numbers, so none were exposed, and that it has not detected impact to its Inference, Training SDK and Nexus products, which it says are zero data retention by default.
Fireworks says the evidence indicates links to a financially motivated cybercriminal group that aims to sell tokens from unauthorized access to large language models to illicit routing gateways, a practice known as "LLMJacking." The company has not named the group, and eSecurityPlanet notes the evidence does not establish that the same attackers were behind both October incidents. It says it notified law enforcement and is working with Palo Alto Networks' Unit 42 on the investigation, which it describes as ongoing. It was Fireworks' second disclosed incident in about a month: in early October, attackers accessed environment variables holding API keys and tokens for a limited number of accounts, BleepingComputer reports.
What should Notion users do?
According to BleepingComputer, Notion is telling affected users to change any passwords, API keys or other credentials they typed into the affected chats. Workspace owners who may be affected can review the data through a security incident review in Notion's desktop web interface. Users who have had no notice, and whose workspace owner has had none either, do not need to do anything for now and are most likely not in the affected group, Notion says.
Questions readers are asking
Was Notion itself hacked?
Notion says the transcripts were exposed through a breach at Fireworks AI, a vendor that runs some of the open-weight models in Notion AI and analyzed conversations for the Insights feature. None of the cited reports describes a breach of Notion's own systems.
How many Notion users are affected?
Notion says fewer than 1% of its users are affected, according to BleepingComputer.
How do I know if I am affected?
Notion says that if neither you nor your workspace owner has received a notice from it, no action is needed for now and you are most likely not part of the affected group.
What should affected users do?
Notion recommends changing any passwords, API keys or other credentials typed into the affected chats.
Sources
- #notion
- #fireworks-ai
- #data-breach
- #supply-chain
- #ai-security
- #llmjacking
Get the day's top AI & tech stories on WhatsApp: Follow the NextFish channel
Get stories like this every morning.
Related
Cybersecurity4 min
ConnectWise urges ScreenConnect client updates as SCKit targets technicians
Microsoft, Huntress and Blackpoint have analysed the toolkit used against MSPs. It is built to jump to technicians, but researchers say they have not seen that work.
Cybersecurity3 min
Landmark Group says Shukran loyalty members' personal data was accessed
Names, contact details and dates of birth were involved, the Dubai retailer says, but not card details, passwords or points. It has not said how many members were affected.
Cybersecurity3 min
Operation Endgame takes down Gootloader, CastleRAT and StealC servers
The Justice Department says Gootloader infected at least 25,000 computers and was used against at least 44 US ransomware victims. Europol counts 235 servers hit in all.
Cybersecurity2 min
TSMC says leaked employee data matches records at a cloud provider
The chipmaker says its own systems show no sign of compromise after a seller offered a 6.9GB file of employee records on hacker forums for three bitcoin.