Client->server communication using json, silverlight and php

PuzzleShare
Submit to StumbleUpon

Hi and welcome back to my blog!

This time i’m going to talk about some of the technology we created for our game PuzzleShare; the client->server communication system. The system we created uses silverlight on the client, but the code and techniques are applicable to all .net languages.

PuzzleShare

In PuzzleShare we use this system for all communication to the server. We log new users in our mysql database, we track each users progress, log scores, let them save their own levels and play other peoples levels. None of this would have been possible without this system.

Danger Will Robinson, Danger!

Of course, you never ever want to directly access your database from the client for obvious security reasons, so it becomes a requirement that you proxy all your requests through a server first and hence this article.

Architecture

You might well ask: ‘why didn’t you just use WCF services?’. Well, the answer is simple: our servers are all LAMP based, not windows; so short of trying to get mono up and running on them, we needed a different solution.

Existing solutions

Being conscientious little developers we of course scoured the net for existing plug and play solutions and quickly came across WebOrb which, although large (tens of megabytes of source) seemed a great fit for our needs.

Things went well at first, but then i started running into problems with the installation not including the silverlight client, then once that was resolved i started getting strange crashes on the client side (which was undebugable since you aren’t given their client source)… The final nail in the coffin was when i sent an array of enum values to the server and found that it had just converted all the values into the enum string names instead of leaving them as ints!

This became necessary

A new way, but which way?

So, obviously this wasn’t going to do; we needed a new solution to this. The question then became which technology to adopt.

Requirements

  • Complex type serialisation and de-serialisation on both client and server
  • Ability to call multiple different server side scripts and individual functions within those scripts
  • Compact message representation
  • Message hashing to protect the content from man-in-the-middle attacks

Data encoding

Ideally, we encode all our messages in binary and send them that way, but we didn’t have the amount of time available to us which would be required to implement such a feature on both server and client, so we had to adopt another encoding.

Two existing data encoding technologies quickly became apparent. XML and JSON. We looked briefly at XML but its quite verbose in its specification so that was quickly ruled out in favour of JSON, which is much more compact. Here is the same data encoded both ways:

{
    "glossary": {
        "title": "example glossary",
		"GlossDiv": {
            "title": "S",
			"GlossList": {
                "GlossEntry": {
                    "ID": "SGML",
					"SortAs": "SGML",
					"GlossTerm": "Standard Generalized Markup Language",
					"Acronym": "SGML",
					"Abbrev": "ISO 8879:1986",
					"GlossDef": {
                        "para": "A meta-markup language, used to create markup languages such as DocBook.",
						"GlossSeeAlso": ["GML", "XML"]
                    },
					"GlossSee": "markup"
                }
            }
        }
    }
}
<!DOCTYPE glossary PUBLIC "-//OASIS//DTD DocBook V3.1//EN">
 <glossary><title>example glossary</title>
  <GlossDiv><title>S</title>
   <GlossList>
    <GlossEntry ID="SGML" SortAs="SGML">
     <GlossTerm>Standard Generalized Markup Language</GlossTerm>
     <Acronym>SGML</Acronym>
     <Abbrev>ISO 8879:1986</Abbrev>
     <GlossDef>
      <para>A meta-markup language, used to create markup
languages such as DocBook.</para>
      <GlossSeeAlso OtherTerm="GML">
      <GlossSeeAlso OtherTerm="XML">
     </GlossDef>
     <GlossSee OtherTerm="markup">
    </GlossEntry>
   </GlossList>
  </GlossDiv>
 </glossary>

The density of the JSON is much less; the reason being it doesn’t require the verbose closing tags that XML does. In fact, once all the carriage returns, tabs and spaces are removed from both examples, JSON is 77% the byte-size of the XML.

Components

Now that we had decided on JSON as our encoding, we just need to assemble rest of the puzzle.

Once we started looking into this we quickly discovered that a lot of the component parts required to solve this problem already existed:

Silverlight

System.Net.WebClient provides a very simple interface to POST data to the server and receive the response in an asyncronous fashion.

System.Runtime.Serialization.Json.DataContractJsonSerializer provides built in complex type serialisation and de-serialisation.

These two require the following references be added to your project: System.ServiceModel.Web and System.Runtime.Serialization.

PHP

json_encode and json_decode are built in functions which serialise and deserialise complex types in php.

Now we were set to do the actual implementation.

Client side

The first thing we needed to was to consider what the interface would actually be like on the client; i.e. how would we be sending requests and getting responses back?

The first thing we’d need would be a proxy class to contain state information about the end-point on the server through which we’d be make the requests.

// make a proxy with the correct end-point
Proxy proxy = new Proxy( new Uri("http://paul.wildbunny.co.uk/jsonTest/proxy.php") );

Then once we had that, how might our requests look?

// fire the request
proxy.Request("test.php", "TestProxy::DoTest", new object[]{parameter}, OnResponse, OnError, state);

Here, we are specifying the actual script to call on the server, the class name and function, any parameters that we want to pass, our response handler, error handler and any state information that we’d like to be able to access in the response/errors handlers.

Great! Looks nice and simple; lets dig into it some more…

In order to actually POST some data from the client to the server we can use the System.Net.WebClient class, which exposes easy to use functionality:

using System;
using System.Net;
using System.Windows;
using System.Windows.Controls;
 
namespace clientServer.Code
{
	public class SimpleWebClientExample
	{
		public SimpleWebClientExample(Uri endpoint, string stringToSend)
		{
			WebClient client = new WebClient();
			client.Encoding = System.Text.Encoding.UTF8;
			client.Headers[HttpRequestHeader.ContentType] = "application/json";
			client.UploadStringCompleted += new UploadStringCompletedEventHandler(OnUploadStringCompleted);
 
			try
			{
				client.UploadStringAsync(endpoint, stringToSend);
			}
			catch (Exception)
			{
			}
		}
 
		void OnUploadStringCompleted(object sender, UploadStringCompletedEventArgs e)
		{
			throw new NotImplementedException();
		}
	}
}

This simple example just sends the given string to the end-point on the server and receives the result. Setting the content-type is one of the important things to watch out for forgetting to do; in this case its application/json since we’re sending JSON data.

Ok, and what about de/serialisation?

using System;
using System.Net;
using System.Windows;
using System.Windows.Controls;
using System.Text;
using System.IO;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Json;
 
namespace clientServer.Code
{
	[DataContract]
	public class TestClass
	{
		[DataMember]
		public int m_count;
 
		[DataMember]
		public string m_name;
	}
 
	public class SimpleSerialisationExample
	{
		public SimpleSerialisationExample(string jsonString)
		{
			TestClass obj = Deserialize<TestClass>(jsonString);
		}
 
		private T Deserialize( string jsonString )
		{
			byte[] buffer = UTF8Encoding.UTF8.GetBytes(jsonString);
 
			using( MemoryStream jsonStream = new MemoryStream(buffer) )
			{
				DataContractJsonSerializer serialiser = new DataContractJsonSerializer( typeof( T ) );
 
				return ( T )serialiser.ReadObject( jsonStream );
			}
		}
	}
}

This one is a little more complex; the meat of the task is DataContractJsonSerializer which does the actual hard work. In order for it to be able to function it needs to have [DataContract] and [DataMember] tags added to the complex type which needs de/serialising. In this example we are have create the class TestClass which we like to de-serialise.

Because it needs to work with a stream we must convert our json string into a stream before calling serialiser.ReadObject().

Serialisation is similar:

private string Serialise( object obj, List<Type> knownTypes )
{
	// make one
	DataContractJsonSerializer serializer = new DataContractJsonSerializer( obj.GetType(), knownTypes );
 
	// serialse the object
	MemoryStream stream = new MemoryStream();
	serializer.WriteObject(stream, obj);
 
	// return the string
	return UTF8Encoding.UTF8.GetString(stream.GetBuffer(), 0, (int)stream.Length);
}

The slight caveat here is that the serialiser needs to know about any custom types used in the object, so these must be passed in as well.

Packaging

In order for the server to understand which script to call, which function and what parameters to use, we have to think about how each message will be packaged.

By just using a simple wrapper to encapsulate all this and then JSON encoding that package we have all we need.

using System;
using System.Net;
using System.Windows;
using System.Runtime.Serialization;
 
namespace clientServer.Code
{
	[DataContract]
	public class ProxyPackage
	{
		public ProxyPackage(string scriptName, string functionName, object[] parameters)
		{
			m_scriptName = scriptName;
			m_funcitonName = functionName;
			m_parameters = parameters;
		}
 
		[DataMember]
		public string m_scriptName;
 
		[DataMember]
		public string m_funcitonName;
 
		[DataMember]
		public object[] m_parameters;
	}
}

The serialiser will be able to convert all this into a JSON string to send to the server. We expect to see something like this:

{"m_funcitonName":"TestProxy::DoTest","m_parameters":[],"m_scriptName":"test.php"}

as the finished request string.

Server side

On the server, our work is even easier:

/* For reference purposes:
 
public class ProxyPackage
{
	public string m_scriptName;
	public string m_funcitonName;
	public object[] m_parameters;
}*/
 
$json = $HTTP_RAW_POST_DATA;
$proxyPackage = json_decode( $json, false );
 
// Validate script
if ( strlen($proxyPackage->m_scriptName) > 0 && file_exists( $proxyPackage->m_scriptName ) )
{
	// Require script
	require_once $proxyPackage->m_scriptName;
 
	// Get user defined function
	$function = $proxyPackage->m_funcitonName;
 
	// call out to function
	$response = call_user_func_array( $function, $proxyPackage->m_parameters );
 
	// Respond to the client
	if ( !FatalErrorHandler() )
	{
		header( 'HTTP/1.1 200 OK' );
		header( 'Content-Type: application/json' );
 
		// must encode even null response otherwise empty arrays do not get encoded correctly
		$jsonResponseString = json_encode( $response );
 
		$jsonResponseString = str_replace( '"json_nan":"json_nan",', '', $jsonResponseString );
		$jsonResponseString = str_replace( '"json_nan":"json_nan"', '', $jsonResponseString );
		echo $jsonResponseString;
	}
}
else
{
	header( 'HTTP/1.1 404 Not Found' );
}

That is pretty much the size of it. The first thing which happens is the JSON string gets deserialsed into the package object – its unfortunate to note at this point that PHP loses the class type information when it deserialises, so you end up with anonymous classes; this is something we’d like to improve on.

After the package is deserialised, we check that the given script exists on the server and we require_once it into the script, which loads the required script. Then we pull the function name out of the package and actually call the function and store the result output by that function.

If there’s not been a fatal error while running the given function, we continue and actually serialise the result before sending it back to the client.

Thats pretty much it!

Example

In this example we’re going to serialise a complex object on the silverlight side, send it to the server, get the result back and display it.

[DataContract]
public struct TestStruct
{
	[DataMember]
	public string m_name;
 
	[DataMember]
	public long m_long;
}
private void OnSendRequest(object sender, System.Windows.RoutedEventArgs e)
{
	// make a proxy with the correct end-point
	Proxy proxy = new Proxy( new Uri("http://paul.wildbunny.co.uk/jsonTest/proxy.php") );
 
	// fire a request to the server, we want to call test.php and within we'd like to call the class'
	// static function TestProxy::DoText, sending it our array of parameters
	TestStruct parameter = new TestStruct{m_name = "Sending to the server", m_long=1000000000005};
 
	// fire the request
	proxy.Request("test.php", "TestProxy::DoTest", new object[]{parameter}, OnResponse, OnError, null);
}

The server side script looks like this:

class TestStruct
{
	public function __construct($name, $long)
	{
		assert( is_string($name) );
		//assert( is_numeric($length) );
 
		$this->m_name = $name;
		$this->m_long = $long;
	}
	public $m_name;
	public $m_long;
}
class TestProxy
{
	static public function DoTest($clientParameter)
	{
		$array =	array
					(
						$clientParameter,
						new TestStruct("hello", 1000000000000001),
						new TestStruct("doctor", 1000000000000002),
						new TestStruct("name", 1000000000000003),
						new TestStruct("continue", 1000000000000004),
						new TestStruct("yesterday", 1000000000000005),
						new TestStruct("tomorrow", 1000000000000006)
					);
 
		return $array;
	}
}

So we expect to see the first of the returned data to be an echo of what we send, proving that the system is working.

Click the ‘Send Request’ button to fire the request off to the server.

Install Microsoft Silverlight

The result sure looks boring, but the components behind it are absolutely essential for the client-server communication requirements of PuzzleShare, and indeed any system where you need to transmit complex types from the client to the server.

Hashing

The astute among you will have noticed i didn’t cover hashing in this article – i’ve run out of time writing this article but if enough people request it by posting in the comments, i will cover hashing at a later date.

Conclusion

The main advantage with this technique over other libraries is that you have total access to all the code, which makes debugging so much easier and you are free to add whatever features you see fit or require. Its also somehow reassuring to be using the built-in libraries to perform all the hard work for you; this means the actual code is tiny – something like 42kb of source code only in this case!

The source

I would absolutely love to make the source-code available for free, but because i need to pay for my food and rent i’ve made it available for a very low cost.  If you like this article, please buy the code and you will help ensure that i’m able to write more like this one.

After purchasing, you will be redirected to a page where you can download the source immediately.

USD 4.99

Subscribers can access the source here

Until next time…

Have fun,

Cheers, Paul.

Submit to StumbleUpon

About Paul Firth

A games industry veteran of ten years, seven of which spent at Sony Computer Entertainment Europe, he has had key technical roles on triple-A titles like the Bafta Award Winning Little Big Planet (PSP), 24: The Game (PS2), special effects work on Heavenly Sword (PS3), some in-show graphics on the BBC’s version of Robot Wars, the TV show, as well as a few more obscure projects.   Now joint CEO of Wildbunny, he is able to give himself hiccups simply by coughing.   1NobNQ88UoYePFi5QbibuRJP3TtLhh65Jp
This entry was posted in JSON, Server side, Silverlight, Technical and tagged , , , , , , . Bookmark the permalink.

17 Responses to Client->server communication using json, silverlight and php

  1. Luben says:

    “The density of the JSON is much less;”
    Just wanted to point out that the density of JSON is higher (datadensity=data/space )

    • Paul Firth says:

      Sorry yes – the density i was referring to was the on screen density in the two examples – they have a comparable number of lines and therefore screen real-estate, yet the quantity of JSON data is less :)

  2. Pingback: Tutorial on client/server communication using JSON and PHP with complex type de/serialisation | www.nalli.net

  3. Pingback: Silverlight PHP kommunikáció | Kerek egy ég alatt

  4. Pingback: Email Marketers need not Fear Honest Client Communication | Support 4 Gurus Blog

  5. [...] Client->server communication using json, silverlight and php | Paul's blog@Wildbunny [...]

  6. Pingback: Nintendo Training 1991 | Support 4 Gurus Blog

  7. Berick says:

    I just wanted to say that I LOVE the fact that you used HAL’s voice test from 2010 as your test data. :D

  8. Ramon Smits says:

    I wanted to point out that xml can be smaller when you make optimal usage of xml attributes. Most data in your example will be available max once. Using attributes saves space as you do not need a closing tag. Besides that, most parsers support values without quotes.

    • Paul Firth says:

      Hi Ramon,

      Yes, this is indeed true using attributes is definitely the way to go if you want to use XML :) We still found that JSON was the better fit for our needs, though :)

      Cheers, Paul.

  9. rahul mishra says:

    hi guys,

    above code is fine work. but I am looking forward to following.

    i am working with silverlight 4.0 and MVC 3.

    i am stuck up at one position i.e. how to pass data from silverlight to MVC controller.

    for ex: consider a login form with username and password as two fields.
    the view is in silverlight and upon clicking the login Command button the logon controller should be called and these two values should be passed to the controller.

    also after login is successful how will the controller call or load the appropriate page?

    if you can help me with this i will be great full to you.

    thanks and regards,
    rahul

  10. Jonathan says:

    Would this work for passing binary data?

  11. Nobody says:

    Execute a user-specified function with user specified arguments… what could possibly go wrong?

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

WP-SpamFree by Pole Position Marketing