Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Page MenuHomePhabricator

Rewrite RCStream in Node.js
Closed, ResolvedPublic

Description

The v0.9 protocol is by far the most popular at the moment, though that is slowly changing now that at least the reference implementation (socket.io-client) has a v1.0 release. However you can't use that yet. And installing the "latest" version of something isn't a good practice anyway. The correct version is now documented https://wikitech.wikimedia.org/wiki/RCStream#JavaScript. The majority of changes between v0.9 and v1.0 is in the backend. The API surface is largely the same and shouldn't block any particular features.

I do however very much want RCStream to be ahead of the curve and encouraging new software to be written in current versions of socket.io and not old ones. RCStream is currently implemented in Python and uses the gevent-socketio package. That package has not yet implemented support for the socket.io v1.0 protocol, so there's no upgrade available. It's been raised upstream (https://github.com/abourget/gevent-socketio/issues/192) however it seems the project is no longer seeing active maintenance.

Considering RCStream is a fairly small library, I'd recommend taking this opportunity to rewrite it in Node.js using the official socket.io server implementation (which is written in node) in favour of using a port.

Event Timeline

Krinkle claimed this task.
Krinkle raised the priority of this task from to High.
Krinkle updated the task description. (Show Details)
Krinkle added a project: EventStreams.
Krinkle added subscribers: Krinkle, ori.

Apparently I did this last year:
https://github.com/Krinkle/node-rcstream

I had forgotten about it though. It looks quite similar implementation wise. I can't stay in what state it is, but it should make for a good starting point.

Krinkle lowered the priority of this task from High to Medium.
Krinkle set Security to None.

This looks like a good opportunity to include diff information. A number of bots are watching the RCStream and immediately fetching relevant diff. This is expensive for everyone, and having a new RCStream include this feature as an option would be great. See T100082

This looks like a good opportunity to include diff information. A number of bots are watching the RCStream and immediately fetching relevant diff. This is expensive for everyone, and having a new RCStream include this feature as an option would be great. See T100082

I appreciate the enthusiasm but this isn't an opportunity for that. The RCStream software is pure middle-ware to publish the internal RCFeed to the outside world using the Socket.IO protocol. It doesn't have access to diffs. It will require added complexity in MediaWiki and RCFeed to make this work via RCStream. RCFeed is not meant to compute or distribute such large blobs.

Let's focus our efforts on RESTBase instead. Its low latency and deterministic scheme will cancel out most reasons for why fetching diffs is unattractive right now. Using RESTBase also requires less engineering effort to implement diffs and provides a more stable, maintainable and extendable platform.

Thanks @Krinkle. It was worth asking... Looking forward to seeing a 1.0 version. Looks like library support in a few languages is good for 1.0.