Threaded Python Websocket Server And Javascript Client

July 23, 2010 at 4:05 pm, Category: Featured, General Computing, Web Development, by Jason

All source code required to begin coding a threaded python websocket server can be found here:

Example Python Websocket Server

The latest draft of the of the WebSocket API has been released by W3 on the 15 June 2010. In the API documentation it details a new technology that exposes the ability for two way communication between web pages and remote hosts. This has been long awaited in the development community and provides exciting new possibilities in terms of real time applications.

Currently, the only way for a client to receive updates from a server is through any given method of polling. The main feature of Websockets is the capability of listening for incoming connections. Full duplex sockets of these kind will reduce bandwidth and server resource overheads and give rise to the possibility of applications such as MMO and first person shooters using soley browser technology. The technology goes hand in hand with the roll out of HTML5 and especially the canvas object and will play a major role in the immenent relandscaping of the Web.

Example Javascript Websocket Client

<script type="text/javascript">
 var ws = new WebSocket("ws://127.0.0.1:1234");
 ws.onopen = function() {
 ws.send("CONNECTED");
 };
 ws.onmessage = function (evt) {
 alert(evt.data);
 };
 ws.onclose = function() {  };
</script>

<button onclick='ws.send("POKE")'>POKE!</button>

The  code above gives a simple example of how to begin communication with your Websocket server. We’re assuming the server is on localhost and listening on port 1234. A new Websocket object is created and a send command is called when the channel is opened. The command “CONNECTED” will be eventually parsed by our server and a response will be sent back that will be caught by and processed by the onmessege function.

Example Python Websocket Server

import websocket

class server:

 conn=0
 users=[]
 socket=0
 uid=0

 def __init__(self,address,port,connections):
 # Point Of No Return!!!
 self.socket = websocket.WebSocket(address, port, connections, self)

if __name__ == "__main__":
 websocketServer = server("127.0.0.1", 1234, 1000)

Here we have the server class and main program. The program simply creates the server which in turn creates the websocket. All the models and preable of whatever application you create with this must be booted up before you enter into the websockets loop.

class user:
 user_id=0
 socket=0
 handshake=0

 def __init__(self, socket, user_id):
 self.user_id = user_id
 self.socket = socket

This class represents a user that would connect through the socket. It contains a reference to the socket channel it has connected on and has a unique id. This class would be extended to be a player, say, if you were developing a game.

import socket
import wsthread
import user
import random

class WebSocket():

 uid=0
 users=[]
 server=0

 def __init__(self, address, port, connections, server):
 self.server = server
 server = socket.socket ( socket.AF_INET, socket.SOCK_STREAM )
 server.bind ( ( address, port ) )
 server.listen ( connections )
 while True:
 channel, details = server.accept()
 self.uid = self.uid + 1
 self.users.append(user.user(channel, self.uid))
 wsthread.WebSocketThread (channel, details, self).start()

In order to get the most out of your Websockets and to allow them to be used for applications such as real time strategy games or MMOs your server must have threading capability. For example, if you were developing a game, this is neccessary for two players to be able move simultaneously. If you are a PHP developer I’m afraid you’re not in luck. There is no ‘real’ threading support for PHP and it is unlikey to be supported in the future. Go here to learn Python.

In this class we open a socket and listen for connections. On accepting an user on the socket we create a unique user id and add them to our array of users. We then start a new thread which will handle that user’s connection with a Websocket. In the following script we see how the Websocket is created through an upgrade handshake of a regular socket connection.

import threading
import hashlib
import socket
import time
import re

class WebSocketThread(threading.Thread):

 def __init__ ( self, channel, details, websocket ):
 self.channel = channel
 self.details = details
 self.websocket = websocket
 threading.Thread.__init__ ( self )

 def run ( self ):
 print ("Monty> Received connection ", self.details [ 0 ])
 self.handshake(self.channel)
 while True:
 self.interact(self.channel)

 def finduser(self, client):
 for user in self.websocket.users:
 if user.socket == client:
 return user
 return 0

 def send_data(self, client, str):
 str = b"\x00" + str.encode('utf-8') + b"\xff"
 try:
 return client.send(str)
 except (IOError, e):
 if e.errno == 32:
 user = self.finduser(client)
 print ("Monty> pipe error")

 def recv_data(self, client, count):
 data = client.recv(count)
 return data.decode('utf-8', 'ignore')

 def get_headers(self, data):
 resource = re.compile("GET (.*) HTTP").findall(data)
 host = re.compile("Host: (.*)\r\n").findall(data)
 origin = re.compile("Origin: (.*)\r\n").findall(data)
 return [resource[0],host[0],origin[0]]

 def handshake(self, client):
 shake = self.recv_data(client, 255)
 headers = self.get_headers(shake)
 our_handshake = "HTTP/1.1 101 Web Socket Protocol Handshake\r\n"+"Upgrade: WebSocket\r\n"+"Connection: Upgrade\r\n"+"WebSocket-Origin: "+headers[2]+"\r\n"+"WebSocket-Location: "+" ws://"+headers[1]+headers[0]+"\r\n\r\n"
 client.send(our_handshake.encode('latin-1'))

 def interact(self, client):
 users = self.websocket.users
 this_user = self.finduser(client)
 data = self.recv_data(client, 255)
 print (data)
 if(data[1:]=="CONNECTED"):
 self.send_data(this_user.socket, "Welcome")
 if(data[1:]=="POKE"):
 self.send_data(this_user.socket, "Quit poking me!")

This is the important bit and where the fun begins. When the connected client is handed off to the thread the first thing that happens is that the details of the connection are printed on the command line. If you are running your server from a command line you will see this when you run your javascript client.

Importantly, the server must now perform the upgrade handshake that will promote the socket to a Websocket. On connection the client sends data from which you must extract the resource, host and origin. The function get_headers performs this. This data is passed to our_handshake. This will allow for your server to work from any incoming address. I’ve seen a lot of early examples where people are hardcoding localhost into this part, so this is essential for getting this working online.

Now we’ve responded with the upgrade handshake the thread falls into its own interaction loop. When receiving and sending data the server must wrap and remove the character codes ‘\x00′ and ‘\xff’ as well as utf-8 encoding the data. These prerequisites are outlined in the specification of the API. Now the rest is up to you. In this example the server will respond if sent ‘CONNECTED’ by sending ‘Welcome’ back and then complains when it is poked…

Playing around with this I have managed to get working a realtime ‘click-to-shoot’ 2D tank game with lobby. Opening the game on multiple screens, they updated instantly and smoothly. It was an amazing feeling and very exciting. I would be eager to hear any implemenations you come up with!

Edit 04/01/2011 Sec-WebSocket-Key Implementation

As of the 14/12/2010 the websocket specification changed to include a security measure to ensure that non websocket requests could not interfer with your connection. The code has since been updated to use this security measure. The following is an exerpt from the wsthread.py file that will handle this:

def get_headers(self, data):
 bytes = data[len(data)-8:]
 data = data.decode('utf-8', 'ignore')
 resource = re.compile("GET (.*) HTTP").findall(data)
 host = re.compile("Host: (.*)\r\n").findall(data)
 origin = re.compile("Origin: (.*)\r\n").findall(data)
 key1 = re.compile("Sec-WebSocket-Key1: (.*)\r\n").findall(data)
 key2 = re.compile("Sec-WebSocket-Key2: (.*)\r\n").findall(data)
 return [resource[0],host[0],origin[0],key1[0],key2[0],bytes]

 def part(self, token):
 digits=""
 for d in re.compile('[0-9]').findall(token):
 digits = digits + str(d)
 count=0
 for s in re.compile(' ').findall(token):
 count = count + 1
 return int(int(digits)/count)

 def handshake(self, client):
 shake = self.recv_data_unencoded(client, 255)
 headers = self.get_headers(shake)
 challenge = pack('>II8B', self.part(headers[3]), self.part(headers[4]), headers[5][0], headers[5][1], headers[5][2], headers[5][3], headers[5][4], headers[5][5], headers[5][6], headers[5][7])
 hash = hashlib.md5(challenge).digest()
 our_handshake = "HTTP/1.1 101 Web Socket Protocol Handshake\r\n"+"Upgrade: WebSocket\r\n"+"Connection: Upgrade\r\n"+"Sec-WebSocket-Origin: "+headers[2]+"\r\n"+"Sec-WebSocket-Location: "+" ws://"+headers[1]+headers[0]+"\r\n\r\n"
 client.send(our_handshake.encode('latin-1') + hash)

To walk you through this the new sent header will contain 3 extra parts.

1. The first key
2. The second key
3. 8 random bytes at the end

The keys contain a encoded number, which is the concatination of the numbers within them divided by the number of spaces within them. This is given by the part function. You then need to construct a list of 16 bytes using the pack function containing the first part no. as a list of bytes, the second part no. as a list of bytes and the orginal list of bytes. This is then put through an md5 hash and tacked onto the end of the handshake response.

I feel this is overly complex for what the purpose of the security hash does but rewarding to crack none the less!

Bookmark It