activa's blog

.NET, Web, Mobile and more stuff I can't stop talking about

Handling dates in JSON responses is something many web developers struggle with. The JSON Specification doesn’t specify how dates should be represented in a JSON string, so every implementation invented its own way of representing dates.

These are some of the formats in use today:

  1. {“date”: new Date(ms_since_epoch) }
  2. {“date”: Date(ms_since_epoch) }
  3. {“date”: “Date(ms_since_epoch) }
  4. {“date”: “/Date(ms_since_epoch)/” }
  5. {“date”: “\/Date(ms_since_epoch)\/” }
  6. {“date”: “yyyy-MM-ddTHH:mm:ssZ” }
  7. {“date”: “yyyy-MM-ddTHH:mm:ss” }

Formats 1 and 2 are actually invalid JSON. However, they’re the only formats that were handled correctly by jQuery 1.3.2 and earlier. eval() also handles these. But again, it’s not valid JSON.

That leaves the other 5 formats, which are valid according to the JSON specs.

Starting with version 1.4, jQuery’s built-in JSON evaluator will check if there is a function JSON.parse() available. If it is there, it will use that function to evaluate JSON objects. If not, jQuery will use the “unsafe” eval() way of parsing JSON. JSON.parse() is a function from the json2.js file, which can be downloaded from the JSON website

The problem

JSON.parse() doesn’t handle dates. At all. But it does have a way to handle non-standard values by specifying an extra parameter “reviver”, which is a function that takes a value and returns the value converted to the data type of your choice.

For example: if you want to handle the ISO date format correctly, you can do this:


var parsedObject = JSON.parse(jsonData, function (key, value) {
   var a;

   if (typeof value === 'string') {
     a =/^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2}(?:\.\d*)?)Z$/.exec(value);

     if (a) {
       return new Date(Date.UTC(+a[1], +a[2] - 1, +a[3], +a[4], +a[5], +a[6]));
     }
   }

   return value;
});

(this example is taken from the json2.js source code)

This function will check if a string value parsed from the JSON string is in a specific date format, and return a Javascript date object.

You could expand this function to handle every other possible date format, which shouldn’t be too hard to do, but how do you tell jQuery to use this “reviver” function?

Well, you can’t.

The solution

There’s an easy solution: after including json2.js, redefine the JSON.parse() function so it passes your conversion (reviver) function to the original JSON.parse() function:

<script type="text/javascript" src="json2.js"></script>
<script>
(function() {
   var _origParse = JSON.parse;

   JSON.parse = function(text) {
      return _origParse(text, function(key, value) {
         var a;

         if (typeof value === 'string') {
            a = /^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2}(?:\.\d*)?)Z$/.exec(value);

            if (a)
                 return new Date(Date.UTC(+a[1], +a[2] - 1, +a[3], +a[4], +a[5], +a[6]));

            if (value.slice(0, 5) === 'Date(' && value.slice(-1) === ')') {
               var d = new Date(value.slice(5, -1));

               if (d)
                  return d;
            }
         }

         return value;
      });
   }
})();

</script>

Now when you return some JSON object to your jQuery script, dates will be parsed correctly, without having to change your code. The code snippet above handles cases 3 to 6 correctly. I’ll leave it up to you to add case 7…

The iPhone is a fabulous device, few people will argue about that.

But…

When I started developing iPhone apps about a year ago, it felt… awkward. If you want to create an app for the iPhone, you have to use Objective C, which was invented sometime around 1934. I’m kidding of course: it was 1986, but that’s a small detail.

Apart from the odd syntax (which is fine, other languages have odd syntax as well), I was amazed by the lack of basic language and framework features we take for granted these days.

To illustrate the contrast, I’ll show a small function that strips the time portion of a date, written in Objective C:

+ (NSDate *) stripTime:(NSDate *) date {
   NSCalendar *gregorian =
         [[NSCalendar alloc] initWithCalendarIdentifier:NSGregorianCalendar];

   NSDateComponents *components =
          [gregorian components:(NSYearCalendarUnit | NSMonthCalendarUnit | NSDayCalendarUnit)
                       fromDate:date];

   date = [gregorian dateFromComponents:components];

   [gregorian release];

   return date;
}

The same thing in C#:

public DateTime StripTime(DateTime date)
{
    return date.Date;
}

I guess I have been spoiled by C#. Can you imagine what it would be like to consume a SOAP webservice from Objective C? Well, check this out. (I have to admit: it received a 5-star rating, so I guess it was worth the 62 hours spent on it)

PS. I realize I suck at Objective C, and I’m sure a better Objective C developer can shave off a line or 2 of my code, but you get the picture…

Thank god a few geniuses over at Novell created MonoTouch

I’ve been developing iPhone apps for the past 9 months using the recommended development tools, being Xcode, Interface Builder and the iPhone SDK.

Being a C# .NET programmer didn’t make this easy: Objective-C is… weird and… primitive. Dealing with pointers after 10 years of focusing on problems instead of memory addresses is a slight culture shock, but luckily having previous C and C++ experience helps a lot.

In September last year, the geniuses (by lack of a stronger word) at Novell’s Mono team came up with MonoTouch, a C# cross-compiler that allows you to build iPhone apps using .NET and C#, including the use of most of the .NET Framework. I’m not going to go into detail on how this works. Enough has been written about this subject.

One feature of MonoTouch caught my attention: as of version 1.2, an ADO.NET provider was added for accessing the iPhone’s native SQLite database format. A relational database accessed from C# code desperately needs an ORM, don’t you agree? So last weekend I decided to try and port the CoolStorage ORM to MonoTouch. It wasn’t as easy as I thought, because of some severe limitations in the version of SQLite installed on the iPhone, but in the end I got it working, and working well.

To give you an idea of the time you’ll save by using an ORM like CoolStorage on the iPhone, I will present three examples:

  1. Reading a list of records using the iPhones’s SQLite library (C / Objective-C)
  2. Reading a list of records using ADO.NET + MonoTouch
  3. Reading a list of records using CoolStorage + MonoTouch

We’re not going to make it too complicated. The example reads a list of customer records. It retrieves the ID, Name and City. No filters are being applied, simply a list of customers ordered by Name. After that, it prints the records to the console.

The examples also omits the declaration of the Customer class (which is used to hold a single customer record)

Native Objective-C example:

NSArray *documentPaths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *documentsDir = [documentPaths objectAtIndex:0];

databasePath = [documentsDir stringByAppendingPathComponent:@"mydb.db3"];

sqlite3 *database;

customers = [[NSMutableArray alloc] init];

if(sqlite3_open([databasePath UTF8String], &amp;database) == SQLITE_OK) {
   const char *sql = "select id,name,city from customer order by name";
   sqlite3_stmt *stmt;

   if(sqlite3_prepare_v2(database, sql, -1, &amp;stmt, NULL) == SQLITE_OK) {

      while(sqlite3_step(stmt) == SQLITE_ROW) {
           Customer *customer = [[Customer alloc] init];

           customer.CustomerID = sqlite3_column_int(stmt, 0)];
           customer.Name = [NSString stringWithUTF8String:(char *)sqlite3_column_text(stmt, 1)];
           customer.City = [NSString stringWithUTF8String:(char *)sqlite3_column_text(stmt, 2)];

           [customers addObject:customer];

	   [customer release];
      }
   }

   sqlite3_finalize(stmt);
}

sqlite3_close(database);

for (int i=0;i&lt;[customers count];i++) {
   Customer *customer = [customers objectAtIndex:i];

   NSLog("ID: %d, Name: %@, City: %@",customer.CustomerID,customer.Name,customer.City);
}

It’s pretty obvious that things will get unbearably complicated when you try to do things like accessing related records, persisting data to the database, and so on.

Now let’s introduce MonoTouch and ADO.NET.

ADO.NET example (MonoTouch):

string dbName = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Personal), "mydb.db3");

var conn = new SqliteConnection("Data Source=" + dbName);

List customers = new List();

using (var cmd = conn.CreateCommand())
{
  conn.Open();
  cmd.CommandText ="select id,name,city from customer order by name";
  using(var reader = cmd.ExecuteReader())
  {
    while (reader.Read())
    {
      Customer customer = new Customer();

      customer.CustomerID = int.Parse(reader["id"].ToString());
      customer.Name = (string) reader["name"];
      customer.City = (string) reader["city"];

      customers.Add(customer);
    }
  }
}

foreach (var customer in customers)
   Console.WriteLine("ID: {0}, Name: {1}, City: {2}",customer.CustomerID,customer.Name,customer.City);

Now that’s a lot better, don’t you think? But still, it requires a lot of boilerplate code that’s just there to distract you from what you’re actually trying to do: read a list of records from the database.

So let’s move to the next level:

CoolStorage example (MonoTouch):

string dbName = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Personal), "mydb.db3");

CSConfig.SetDB(dbName, false);

var customers = Customer.List().OrderedBy("Name");

foreach (var customer in customers)
   Console.WriteLine("ID: {0}, Name: {1}, City: {2}",customer.CustomerID,customer.Name,customer.City);

Well, you get the picture… Now you can focus on the application instead of data access technology. Isn’t this what it’s all about?

In case you want to use this in your own code, go ahead and grab the latest development build of CoolStorage, which now fully supports MonoTouch. It can be downloaded from the CoolStorage website.

Best of all, it’s open source…

Read till the end, because nothing is what it seems

What is equality?

Equality means different things in different programming languages. In most modern languages (C# for example), you can define your own terms of what equality means. In others you can’t.

Take javascript for example. There are actually two kinds of equality: normal equality and being identical.

To check if two variables or values are equal, you use:

f (a == b) {
  // ...
}

To check if two variables are identical, you use:

if (a === b) {
  // ...
}

Most articles (and even supposedly good books) define the latter comparison as being equal and of the same type. While this over-simplification may be true in the majority of cases, it is very inaccurate, because you have to know what “equal” means. I’m not going to talk about javascript’s normal equality operator (==), because enough has been written about that. I’ll focus on the === operator.

The correct meaning of === is that the 2 operands should be identical. This means they should reference the same object, or, in the case of value types, the values should be the same. Only numbers and booleans are value types. Strings behave as value types, but they are actually reference types.

Bringing this all together, let’s look at some examples of equality in javascript:

var a = 1;
var b = 1;

alert(a == b); // true
alert(a === b); // true

b = "1";

alert(a == b); // true
alert(a === b); // false

These are the obvious ones. Now the more interesting stuff:

var a = [1,2,3];
var b = [1,2,3];
var c = a;

alert(a === b); // false (these look equal to me, and of the same type)
alert(a === c); // true

Remember I mentioned that strings behave like value types? Now it becomes interesting:

var a = "12" + "3";
var b = "123";

alert(a === b); // returns true, because strings behave like value types

And to make it really interesting:

var a = new String("123");
var b = "123";

alert(a === b); // returns false !! (but they are equal and of the same type)

I thought strings behave like value types? Well, it depends who you ask…

EDIT: As one of the commenters (zihotki) points out, the last example adds another twist to the story. Creating a string using the String() object constructor actually doesn’t create a variable of type “string”, but of type “object”. But you can use the object as a string. If you are unaware of this behavior, you can easily shoot yourself in the foot without knowing it.

Even before there was talk about ASP.NET MVC, there were a few open source MVC frameworks available for .NET. Among them, MonoRail was best known, but there was also ProMesh.NET, a .NET MVC web framework that evolved from a small personal project to a full-blown MVC framework for building web applications in .NET, without using ASP.NET WebForms.

Vici Project

A few months ago, just before the release of ProMesh.NET version 2.0, we decided to move the project to the Vici Project, a project that bundles several open-source frameworks for .NET 2.0. The idea of the Vici Project is to provide .NET developers with a collection of lightweight libraries and frameworks, and at the same time get the community involved in the development and support of these libraries.

To make it easier to get the community involved, a complete system was created with the following features:

  • Central SubVersion repository with online browsing (using WebSVN)
  • Automated build server (using JetBrains TeamCity)
  • Wiki infrastructure for maintaining documentation
  • Support forum

After several months of testing all of this, the project is finally ready to go live. There is still a lot of work to be done, especially on documentation, but I think there is no point in postponing the release.

The first sub-project of the Vici Project to be released is Vici MVC, formerly known as ProMesh.NET. Later this week, 2 other projects will be released as well: Vici Parser (formerly LazyParser.NET/SharpTemplate.NET) and Vici CoolStorage (formerly CoolStorage.NET)

    Vici MVC 2.0 new features (compared to ProMesh.NET 1.2)
  • New powerful URL routing engine. Also supports “extension-less” URLs with IIS 7.0 or IIS 6.0 (with wildcard mapping or URL rewriting)
  • Support for view components (“inline” controllers with templated views)
  • Support for sub-templates with parameters
  • Support for template macros
  • Configurable template syntax
  • Full C# 2.0 expression supported in templates
  • New template language syntax (the old one is still supported)

Anyone interested in contributing to the Vici Project, please let me know by posting on the forum (or by sending me a private message on the forum. My username is activa)

Early next year, a cool new service will go online targeted at web developers and designers. It will go by the name of dStyler.com but it’s a little hard to explain what the service will do, but I’ll start with a simple example:

It turns this into this
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed vel arcu eget lorem dapibus molestie.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed vel arcu eget lorem dapibus molestie.
With just a single line of CSS

What dStyler actually provides is real-time online image generation. It is a highly scalable REST-style webservice that generates images based on a custom-built URL. For example, the box on the right is generated by the following url:

http://dstyler.com/ufnqfr/204×254/bo-333/ib-fff-2/ro-5/gr-eef-b8e/gr-fff9-fff0-210×260/sh-0008-0-r3.jpg.

It is in a way similar to the Google Charts API and the Google static maps API.

Some of the things dStyler can generate:

  • Solid backgrounds
  • Gradients
  • Single or double borders
  • Drop shadows
  • Image embedding (useful for watermarks)
  • Special effects
  • Rounded corners
  • Slicing

The online administration interface allows you to:

  • Build images in an interactive way (to generate the URL for you)
  • Upload images to your account that can be used for generating images

Best of all, dStyler.com will be a free service

We are currently looking for people interested in beta testing this service. So if you’re interested, contact me at beta -at- dstyler -dot- com (I know, I hate these obfuscated e-mail addresses more than anyone, but sadly enough, the world is a nasty world filled with f**king spammers)




This weblog is sponsored by The Vici Project.

We all love unit tests (do we?). Yes we do (repeat 100 times). And we all love mocking for making life easier, but sometimes we just want to write some simple unit tests that don’t require any mocking at all, because the code you are testing is easy to test without having to resort to all those mocking tricks

But what if you need to test code that relies on the date or some elapsed time? Let’s say you wrote a simple caching class that automatically removes items from the cache when a certain amount of time has passed since the last access. It’s tempting to slap some Thread.Sleep() calls in there to trigger expiration of cache items, but that will slow down your unit test, and most importantly, it’s not accurate. What if you want to test some edge cases? Like, what happens when you access an item at the exact timeout period?

A few years ago I worked with a brilliant Java architect who simply said: “just mock the time!”. What he meant was that I should create a fake DateTime class which behaves like the real DateTime class with one important difference: YOU control the time, not the system clock.

How is that done?

Simply create an interface with a single getter property of type DateTime:

public interface ITimeProvider
{
   DateTime Now { get; }
}

Then you create 2 classes that implement this interface: one that returns the real time, and one that can be used to control a “fake” time:

public class RealTimeProvider : ITimeProvider
{
    public DateTime Now { get { return DateTime.Now; } }
}

public class MockTimeProvider : ITimeProvider
{
    private DateTime _time;

    public DateTime Now { get { return _time; } set { _time = value; } }
}

Of course the class(es) you want to test should be aware of this. For example, our cache class could look like this:

public class Cache
{
     private ITimeProvider _time = new RealTimeProvider();

     public ITimeProvider TimeProvider { get { return _time; } set { _time = value; } }

     // ... the rest of the class implementation
}

In our class implementation, all calls to DateTime.Now should be replaced by TimeProvider.Now.

And now the fun part, faking the time in our unit tests:

[Test]
public void TestCacheTimeout()
{
    ITimeProvider time = new MockTimeProvider();
    Cache cache = new Cache();

    cache.TimeProvider = time; // Tell our cache class to use our fake time

    cache.Add("A" , 1); // add an item to the cache

    Assert.IsTrue(cache.Contains("A")); // check if it's added

    time.Now += TimeSpan.FromHours(1); // let one hour go by...

    Assert.IsFalse(cache.Contains("A")); // check if it was automatically removed
}

That’s pretty cool and simple, isn’t it? Wouldn’t it be nice if we could do the same in real life? ;-)

Next week I’ll talk a little about unit testing multithreaded concurrency issues…

Jeff Atwood (Coding Horror) wrote an interesting blog post about detecting hyperlinks in blocks of “regular” text. I was suprised by the wave of negative comments he received, mainly because Jeff tried to solve a complex problem with a “simple” regular expression. I don’t agree with these comments at all. Software developers are too obsessed with borderline cases. What’s wrong with “good enough”? What’s wrong with solving 99.99% of all possible situations? After all, what’s the worst that can happen if the algorithm is not 100% correct? You’ll see a bad hyperlink. Big deal…

Anyway, the thing is, as Jeff points out, URL’s can contain some pretty weird characters you wouldn’t expect, like ‘(‘, ‘)’ and ‘,’. This can become quite frustrating when trying to parse something like this:

My website (at http://www.mysite.com/coolpage) is becoming very popular, especially because of traffic coming from http://www.othersite.com/coolestpage, which is a cool site I recently found.

For humans, it’s pretty obvious where the hyperlinks are, but what if the links contain some of the non-standard characters I mentioned above? For example:

http://www.mysite.com/coolpage(nice).aspx or http://www.othersite.com/coolest,wildestpage. Both of these are valid URLs.

If you would implement a trivial URL detector, the following URLs would be extracted from the text snippet above:

http://www.mysite.com/coolpage)” and “http://www.othersite.com/coolestpage,“.

Note the trailing parenthesis and comma. Obviously that is not what we want. The fact is that there is no way we can extract valid URLs by following a set of fixed rules, not even for humans. We humans will “parse” the URLs by looking at the context, but that’s a pretty hard, if not impossible task for software.
What we can do is use some heuristics to satisfy at least 99.99% of all cases, so I started thinking about this a little bit and came up with a solution that uses regular expressions and requires no code at all.

Let’s start with the basics. A URL contains 3 distinct parts, of which the last part is optional:

protocol://host/path

“host” is a hostname (or IP address), optionally followed by a port number, so this is the regex we can use for this:

([a-zA-Z-:@.0-9]+)

Of course, this will not validate the host name part of the URL, but that’s not really our goal now.

Next we should specify what the path looks like. According to the specs, the path can contain any of these characters: letters, digits and any of -;:@&=$_.+!*’,()

In this example, we will only try to match http and https, so our trivial regex would look like:

\b(https|http)://([a-zA-Z-:@.0-9]+)(/[-;:@&=?a-zA-Z0-9$_.+!*',()])?

That will work in most cases, but it fails miserably in our text snippet example.

Let’s make an assumption: we assume that a URL containing parentheses always has matching parentheses, as in “http://www.mysite.com/Some(Nice)Page“. This would exclude URLs with a single parenthesis, so “http://www.mysite.com/Some(NicePage” would not be recognized as a URL. That’s a sacrifice I’m willing to make.

There are several ways of writing a regex like this, but a simple one is:

\b(https|http)://([a-zA-Z-:@.0-9]+)(/((\([-;:@&=a-zA-Z0-9$_.+!*',]*?\))|[-;:@&=?a-zA-Z0-9$_.+!*',]|%\d\d)+)?

Note that I also added some regex code for escaped characters (%nn). Nested parentheses will not be matched.

This would solve the first match in our text snippet, but not the second one with the trailing ‘,’. Again, we will make an assumption: let’s assume that a URL will never end with a period or a comma. I think that’s a pretty safe assumption to make (although technically it is allowed).

Our final regex will then look like:

\b(https|http)://([a-zA-Z-:@.0-9]+)(/((\([-;:@&=a-zA-Z0-9$_.+!*',]*?\))|[-;:@&=?a-zA-Z0-9$_.+!*',]|%\d\d)+)?(?<![,.;])

This regex will match any valid URL pattern, except URLs without matching parentheses. It will also exclude URLs with a period, comma or semicolon at the end.

Again, it’s not a perfect solution, but I bet it will be very hard to find a real-world example where this regex would fail to extract the URL from a piece of text. The regex can still use some tweaking, but you get the picture.

Most people believe dogfooding is the perfect way of ensuring the quality of the software you’re creating. I couldn’t agree more.

The only problem is that when you use your own software, you always have the feeling that it’s not quite “done”. There’s always something that you can do better, always something that can be improved, always some features that you feel would be nice.

Especially when building a framework this can become a real problem because you never feel done. Another problem is making sure your framework maintains backwards compatibility with previous releases.

Well, I just mentioned this because I finally decided to freeze version 2.0 of ProMesh.NET and get it ready for release. Release candidate 3 has just been published on CodePlex

Thoughts on open source frameworks

A few weeks ago, a fellow developer asked me why I chose to make my projects open-source. That was a pretty good question because when you look at it: compared to just building a framework for your own use, it makes life more complicated: you have to write documentation, support users, and more headaches.

I didn’t have to think long before I could answer that question: releasing a custom framework as open source forces you to be disciplined about your code and documentation. After all, you can’t afford to write code you are ashamed of, can you? If you write software for your own use, you tend to write code that… well… sucks.

And of course, let’s not forget the whole point of open source software: letting other developers contribute. It creates a dynamic that you would never get when you’re just building and using your own little framework

If all of the above sounds like a bunch of crap, it’s probably because I’m not a native English speaker, or … it is just crap

A few days ago, I stumbled upon a problem that I never really noticed before Google Chrome was released. It seems that Google Chrome chokes on embedded javascript if some weird bytes are present in the HTML.

It took a while to figure out what was going on. It seems the weird bytes were the byte order mark for UTF-8 documents (hex EF BB BF). When Google Chrome finds these bytes within a <script> block, it will simply stop parsing javascript. No errors, no warnings.

You can argue that this is a problem with Google Chrome, but that still doesn’t explain why these bytes were in my javascript blocks to start with.

The problem is this: I inject javascript in the HTML output from embedded resources. The embedded resources are just .js files created in Visual Studio, marked as “embedded resource”. The resource is then read from the assembly and converted to a UTF8 string and inserted in the HTML script block. The problem is that Visual Studio ALWAYS adds the UTF-8 “byte order mark” when you save a text file. These bytes are also embedded in the embedded resource, which is… annoying.

Of course, you could tell Visual Studio to save your file without signature (“byte order mark”), but you have to do that EVERY time you’ve created a new javascript file. There’s no way to make that the default.

Bummer…

On a positive note, this gives me the perfect excuse to announce the release of Release Candidate 2 of ProMesh.NET 2.0, which has built-in detection for BOM bytes on embedded resources :-)