October 23rd Incident Report

Incident Summary

A third-party vendor, Customer.io, suffered a data breach by a bad actor who targeted their system to steal data and send spam messages. Customer.io provides transactional and marketing email delivery services for Knovio and manages lists of Knovio user names and email addresses. Our account with the vendor was one of several accounts attacked, and as a result the bad actor was able to access a list of Knovio user email addresses, upload a list of new email addresses to target, and send out a series of unauthorized spam messages containing links to weight loss products that appeared to be originating from one of our employees. No customer data, financial information, or user credentials were exposed in this breach.

 

Customer.io has also published a full report regarding this incident: 

Customer.io Breach Report

Timeline

Oct 12, 2019 Attacker gains unauthorized access to Knovio account in vendor’s system

Oct 21-22, 2019 Attacker exports and downloads user list

Oct 22, 2019 23:59 (EDT) Attacker sends first round of spam messages

Oct 23, 2019 12:15 (EDT) Attacker uploads list of new email addresses to target

Oct 23, 2019 2:56 (EDT) Attacker sends second round of spam messages

Oct 23, 2019 7:00 (EDT) Knovio team identifies spam attack in progress originating from compromised account on vendor’s system

Oct 23, 2019 7:05 (EDT) Knovio team resets all user passwords on compromised account

Oct 23, 2019 10:00 (EDT) Knovio team enables 2FA on all user accounts

Oct 23, 2019 10:15 (EDT) Knovio team temporarily disables email sending from account

Oct 23, 2019 10:20 (EDT) Knovio team cycles account API keys

Oct 23, 2019 10:30 (EDT) Knovio team posts status notification of security incident in Knovio application

Oct 23, 2019 10:45 (EDT) Knovio team contacts vendor to report account breach

Oct 23, 2019 13:20 (EDT) Vendor sets its system UI to read-only mode

Oct 23, 2019 13:30 (EDT) Vendor invalidates its system UI sessions, performs additional password reset for compromised accounts

Oct 23, 2019 18:00 (EDT) Vendor sends first communication of incident to all users and customers

Oct 23, 2019 Vendor and Knovio teams perform extensive auditing of user accounts and access

Oct 24, 2019 17:00 (EDT) Knovio team sends email notification of incident to all affected users and customers

Oct 24, 2019 Vendor and Knovio teams perform ongoing investigation and corrective measures

Root Cause

Using an enumeration attack against the signup form and API of the vendor, the attackers identified our Knovio account as a valid account.

The attackers then gained access to our account using a pre-existing password list. The unauthorized access was attained using the correct username and password for the account. It is highly likely that the credentials were obtained from one of the significant breaches, like those collected in combos like Collection #1, Anti Public, or http://exploit.in.

Our user account did not have two-factor authentication (2FA) enabled as an extra security layer.

The attackers used the access they obtained to export and download lists of Knovio user names and email addresses, upload a list of new email addresses, and send out a series of unauthorized spam messages.

Data Breach

Neither the Knovio application itself, nor any Knovio team member’s email account, was attacked or impacted directly by this attack, and no customer data hosted in the Knovio system was compromised. The attack also did not target any system containing financial or billing data, such as credit card numbers. No user passwords or credentials were exposed in the breach.

 

The data stored in our vendor’s system is limited to email lists that are maintained for transactional and marketing email communications to Knovio users and customers. In the course of the breach, the attacker was able to successfully export and download a list of Knovio users. This data set includes the following user information (as applicable):

  • Email Address
  • First Name and Last Name
  • Organization Name
  • Subscription Information:
    • Account Plan
    • Subscription Name
    • Subscription Period
    • Account Status (active/cancelled)
    • User Type (Student, Teacher, Trainer, etc.)
    • Internal User ID
    • Created Date
    • Last Visit Date
    • Number of Active Presentations
  • IP Address
  • Browser Info

Resolution and Recovery

As soon as we became aware that our account had been compromised and was being used to send large volumes of spam messages, we immediately reset all users passwords, and shortly thereafter reviewed our account access, enabled two-factor authentication, disabled any further sending from the account, and cycled the API keys used by the Knovio application. We then identified the list of users whose data was exported and downloaded by the attackers, and the list of users that had been imported to our account and targeted by the spam campaigns.

 

We began working with our vendor to share details about the attack and coordinate a response. We immediately posted a status notification about the incident on our website and in the Knovio application, and sent out an email communication the following day only after we had fully verified that the threat had been successfully isolated and restored the normal service of our account.

 

We have identified action items for improvement in our password policy, security settings, authentication handling, and other aspects of our security program, and we are actively working with our vendor while their ongoing investigation of the incident continues.

Corrective and Preventative Measures

  • We have enabled a new feature released by the vendor on Oct 24 that makes two-factor authentication mandatory for all users in our account.
  • We have changed the passwords for all other vendor systems to mitigate the risk that any of those systems could be accessed with the same email/password combo used to exploit this account.
  • The vendor is hardening their signup form against this form of attack and working to improve their monitoring and alerting.