Update on security

We were recently contacted by a well-known security consultant, who had managed to create a proof of concept exploit of our authentication mechanism for private channels. We have now mitigated against this exploit both at a service level, and also in a number of our libraries. We have disclosed this information to our customers, and are working closely with a handful of them whose apps could not be secured (due to use of legacy or home-rolled libraries). We are grateful to Egor Homakov for bringing this to our attention, and feel proud of our team for acting swiftly to fix the issue.

We have no reason to believe this exploit has been used maliciously.

Security of our system remains the utmost priority, and we hope that our response demonstrates this commitment. If you have any further questions or concerns about this, please get in touch via support@pusher.com.

Details of the exploit

Our service provides “private” channels that provide an authentication mechanism restricting who can subscribe to them.

We delegate the decision on who can subscribe to which private channels to our customers by having them implement an authentication endpoint that end-users request a token from in order to join a particular channel.

This token is the HMAC signature of the end-user’s connection ID (known as a socket_id) and the channel they wish to join. By presenting Pusher with this signature, they prove that the customer approved the particular combination; only someone in possession of the customer’s secret key could have generated the correct signature.

However, a lack of validation in the libraries we provided to customers lead to a situation where a malicious end-user could submit a malformed socket_id field and have the customer unknowingly sign a string which grants access to private-channel-B even though the end-user appears to be requesting access to private-channel-A. In this way, a malicious end-user with permission to subscribe to one private channel is able to fake permission for any private channel owned by the same customer.

In a similar way, our HTTP API is secured by requiring a signature with each request which could only have been generated by the customer’s secret key. However, again, in limited cases, a malicious end-user might trick a customer into signing a value for socket_id which was sufficient to authenticate an API request to Pusher.

As always with these things, the nitty-gritty is more complicated and involves a few more variables, it’s laid out in detail in Egor Homakov’s blog post

Summary of what we’ve done

• We were made aware of the exploit on the Friday 8th May, and started investigating
• We determined it was a serious threat on Monday, and assigned most of the engineering team to the issue
• We patched our service to be much stricter about the input it received
• We patched the 7 server libraries that we maintain to close the loop
• We contacted and placed on a whitelist our customers whose apps would break under the new validation
• We sent an email to all of our customers encouraging them to upgrade their libraries (sorry for the lack of identification in the subject :) )

What we still plan to do

Next week we will do a full internal post-mortem to examine our response to this issue. We will also dedicate time to helping customers with any problems that crop up.

Lessons learned

Security is hard, and devising your own can be risky. When we originally built it in 2010, there were no existing patterns for securing WebSocket connections. We stand by the model, in its new hardened form, but will also investigate whether it can be further improved.

We care deeply about security, and believe that it is a key benefit of using a hosted service. We patched hundreds of thousands of applications in a couple of days.

Only recently we announced that we encourage everyone to use our SSL endpoints, and that we aren’t charging extra for them.

We’re sorry for any inconvenience this may have caused.