Description
Context
I've been facing some major message corruption issues after upgrading a project to Colyseus 0.16.0.
This app is a typical interactive slideshow where each room hosts 1 master and N participants. The master conducts the session by sending the current slide to participants and participants can interact with the current slide by sending messages back to the master via the WebSocket server.
Bug description
Whenever a participant joins the room, the server broadcasts a "P" message containing the total number of participants in the room.
This is what I'm supposed to see (Colyseus 0.15.57):
A participant joins the room and receives a JOIN_ROOM message:
The number of participants (1) is sent to the master using a "P" message:
This is what I get after upgrading to 0.16.0:
A participant joins the room and receives a JOIN_ROOM packet, this is correct:
The JOIN_ROOM message sent previously leaks into the packet sent to the master, but the size of the packet is correct:
(The extra packet is the master acknowledging the JOIN_ROOM packet that it wasn't supposed to receive).
I initially ruled out Colyseus and ws as being the cause of the issue because I could confirm that the packet were routed correctly but I came back to Colyseus after I realised some work had been done to use a shared buffer for outbound messages a couple months ago. More specifically, this commit: 19f1c3f and a few others around June 12 to June 14, 2024.
I believe the cause of the problem is that buffer.subarray() doesn't create a copy of the buffer and that neither ws nor Node do a copy of the buffer sent by Colyseus and socket.write() isn't synchronous so it will send whatever data is available in the buffer at the moment it is being written to the socket.
Also something to note is that this project uses a HTTPS server in development because I need to perform cross origin authentication and I understand that HTTPS puts more stress on the socket. In fact, I've not been able to replicate the issue with a HTTP server. However, if my suspicion about the send buffer is correct, I don't think there is anything that would shield the HTTP server from that same buffer leak issue in theory because sockets use streams under the hood, which are asynchronous by nature.
Reproduction
A minimal reproducible example is available here.
Steps to reproduce
The steps to reproduce are described in the README on the reproducible example.
Environment & Versions
Colyseus version: 0.16.0+
Node version: 20.19.0, 22.14.0
Operating system: Ubuntu 22, Windows 11