I bet most people don’t have a clue what Vici Core is, and the current website doesn’t offer a lot of insight either, so I thought it might be a good idea to give an overview of the components included in the toolkit. I will follow up with additional blog posts describing the components in more detail.

What is Vici Core?

The most important parts of Vici Core are the dynamic expression parser and template rendering engine. These components allow the developer to incorporate runtime expression parsing and template rendering in their applications. The parser is generic, meaning that it can handle any imaginable syntax, but it includes a C# style expression parser in the core library.
The parser and template renderer are both based on a generic and extensible tokenizer. The tokenizer is also used by the built-in JSON parser.

This is a quick overview of the components in Vici Core:

• Tokenizer
• Expression Parser (C# built-in)
• Template engine
  • Mustache syntax
  • Velocity syntax
  • Vici MVC syntax
• JSON parser
• JSON serializer
• Configuration
• Logging
• Exception-free smart type conversion
• SmartCache
• FastReflection
• Notifications/Subscriptions
• Scheduling
• Binary and Base64 serialization

Best of all, these components can be used on the following platforms:

• .NET 3.5 or higher
• Windows Phone 7.5
• MonoTouch
• Mono for Android

If you want to check it out, feel free to download the source and/or binaries at viciproject.com

Below is a description of most of the components in the framework.

Tokenizer

The Vici Core tokenizer allows you to tokenize any stream of characters to a list of tokens. The tokenizer is extensible so you can define your own tokenizer logic by creating classes implementing the ITokenParser and ITokenProcessor interfaces.

Using the tokenizer, you can take the following string:

Items["Key"] = 45;

And turn it in the following list of tokens:

The code for this is would be:

Token[] tokens = new CSharpTokenizer().Tokenize("Items[\"Key\"] = 45;");

To give you an idea of the way you would write your own token recognizer, this is the source code of one of the built-in classes that will recognize a string (keyword):

    // This class does double duty, implementing both ITokenMatcher and ITokenProcessor
    public class KeywordMatcher : ITokenMatcher, ITokenProcessor
    {
        private int _index;
        private readonly string _string;

        public KeywordMatcher(string s) { _string = s; }

        public ITokenProcessor CreateTokenProcessor()
        {
            return new KeywordMatcher(_string);
        }

        public void ResetState() { _index = 0; }

        public TokenizerState ProcessChar(char c, string fullExpression, int currentIndex)
        {
            if (_index >= _string.Length)
                return TokenizerState.Success;

            return (c == _string[_index++]) ? TokenizerState.Valid : TokenizerState.Fail;
        }

        public string TranslateToken(string originalToken, ITokenProcessor tokenProcessor)
        {
            return originalToken;
        }
    }

Expression Parser

Vici’s expression parser builds an expression tree from any expression. Internally, it tokenizes the input string, reorders the tokens to RPN notation (using the Shunting Yard algorithm) and then creates a true expression tree that can be used to evaluate the expression.

Although you could use Vici’s parser to parse any syntax, only the C# syntax is supported in the core library.

In its simplest form, this is how you could use the expression parser:

CSharpParser parser = new CSharpParser();

int value = parser.Evaluate<int>("5+4*a",new CSharpContext(new {a=10}));

// value == 45

Since it is a true C# expression parser, you can use any C# (2.0) feature you like:

CSharpContext context = new CSharpContext();
CSharpParser parser = new CSharpParser();

context.AddType("Math",typeof(Math));
context.Set("f", new Func<int,int>(i => i*2));
context.Set("obj", new { Value=5 });

int value1 = parser.Evaluate<int>("5 * f(2)", context);
double value2 = parser.Evaluate<double>("Math.Cos(1.0)", context);
string value3 = parser.Evaluate<string>("obj.Value.ToString()", context);

Template Engine

Vici’s template rendering engine is based on the built-in expression parser, adding support for extra control structures like conditional statements, loops and external references.

The templates can be anything from simple strings to HTML files and mail templates. As an example, I’ll present a real-world mail template that could be used to send out order confirmation e-mails to a customer:

Dear $Order.Customer.Prefix $Order.Customer.LastName,

Thank you for your order. The estimated shipping date is $Order.EstShipDate

Your order:
#foreach(item in Order.OrderItems)
$item.Description ($item.Price)
#end

Total price: $ $Order.TotalPrice
...

Rendering this template is done in just a few lines of code:

Order order = OrderService.ReadOrder(...);

CSharpContext data = new CSharpContext(new { Order = order });
VelocityTemplateParser parser = new VelocityTemplateParser();

string renderedMail = parser.Render(template, data);

MailService.SendMail(order.Customer.Email, renderedMail);

This is just one of the many ways you could use the template engine. In addition, it’s very easy to create your own template syntax definitions. Check out the source code of the built-in template syntax handlers to learn more.

JSON Parser

JSON data can be parsed to a dictionary or to a real object. The parser will try to match the JSON data structure to the target object’s properties or fields.

// json:
// { "firstname" : "John" , "lastname" : "Doe", "children" : [ "Kelly","Tim" ] }

public class Person
{
    public string FirstName;
    public string LastName;
    public string[] Children;
}

// ...
JSONParser parser = new JSONParser();

Person person = parser.Parse<Person>(json);

Configuration Framework

The configuration framework allows you to map data from a data source (XML file, app.config, database, …) to class properties and fields.

For example, if this is your app.config file:

<appSettings>
    <add key="Database.Server" value="db.mycompany.com" />
    <add key="Database.Port" value="6005" />
    <add key="Database.Schema" value="Customers" />
    <add key="LogFile" value="Z:\Logs\Logfile.txt" />
</appSettings>

Then you can easily map this to some static “Config” object in your application:

public class Config
{
     public class _Database
     {
          public string Server;
          public int Port;
          public string Schema;
     }

     public static _Database Database;
     public static string Logfile;
}

//...

ConfigManager configManager = new ConfigManager();

configManager.Register<Config>();

configManager.RegisterProvider(new ConfigurationProviderAppConfig());
configManager.Update();

//...

Database.Connect(Config.Database.Name, Config.Database.Port);

The source of your configuration can be anything you like. The framework will call your configuration provider class and the provider simply has to return key/value pairs. Keys can be structured using dotted notation. The values will be converted to the target datatype (strings, numbers, arrays, …)

Versioning is also supported. Your configuration objects will be updated when a version number is changed in your source file.

Logging

Logging is something that can be found in a lot of third-party libraries, but since it’s not that complicated to implement, it was added to Vici Core to minimize dependencies. The end result is a pretty decent logging framework.

It features:

• Pluggable logging providers (logging to multiple providers at the same time)
• Log levels (Debug,Warning,Error,Fatal, …)
• Log rotation and cleanup (by hour,day,week,month,…)
• Dynamic log file names (with embedded dates)

In its simplest form, logging can be added to your application like this:

Logger.Default.AddProvider(new LogProviderFile("c:\\logs\\logfile.txt"));

//...
Logger.Default.Log(LogLevel.Debug, "User logged in: {0}", userName);
//...
Logger.Default.LogException(ex);
//...

SmartCache

What’s in a name? SmartCache is a very simple but flexible caching framework that supports most basic features a caching solution should have:

• Configurable cache size
• Fully typed
• Thread-safe
• Cache entries can be configured to time out after a specific time or at a specific absolute time

Code sample:

var cache = new SmartCache<Record>(500); // max. 500 cache entries

// add item, does not expire
cache.Add(record1.Key, record1);
// add item, expires after 30 minutes when not accessed
cache.Add(record2.Key, record2, TimeSpan.FromMinutes(30));
// add item, expires in exactly 10 minutes
cache.Add(record3.Key, record3, DateTime.Now.AddMinutes(10));

// Retrieving:

// Checks if a record is cached and retrieves it
if (cache.TryGetValue("key1", out record)) ...

// Tries to retrieve a record from the cache, and if not found
// the supplied delegate will be called to fetch the record again
// and store it back in the cache
cache.TryGetValue("key2", out record, () => Database.LoadRecord("key2"));

Type Conversion

The Smart Conversion library allows you to convert any datatype to another data type, without risking type conversion exceptions.

Heuristics are used to convert values from one type to another.

Examples:

string s1 = "25";
string s2 = "0";
string s3 = "";
string s4 = null;
string s5 = "a";
enum LogLevel { Debug = 0, Information = 1, Error = 2, FatalError = 3 };

int a = s1.To<int>(); // returns 25
int b = s2.To<int>(); // returns 0
int c = s3.To<int>(); // returns 0
int d = s4.To<int>(); // returns 0
int e = s5.To<int>(); // returns 0
int? e = s1.To<int?>(); // returns 25
int? f = s2.To<int?>(); // returns 0
int? g = s3.To<int?>(); // returns null
int? h = s4.To<int?>(); // returns null
int? i = s5.To<int?>(); // returns null

LogLevel ll1 = "1".Convert<LogLevel>(); // returns LogLevel.Information
LogLevel ll2 = "3".Convert<LogLevel>(); // returns LogLevel.FatalError
LogLevel ll3 = "Error".Convert<LogLevel>(); // return LogLevel.Error

// If a value can't be converted, the default is returned, which is
// the value 0 (zero) casted to the enum type (according to the C# specs).
// If the target type is nullable and the conversion fails, null is returned.

LogLevel ll4 = "5".Convert<LogLevel>(); // returns LogLevel.Debug
LogLevel ll5 = "Bogus".Convert<LogLevel>(); // returns LogLevel.Debug
LogLevel? ll6 = "5".Convert<LogLevel?>(); // returns null
LogLevel? ll7 = "Bogus".Convert<LogLevel?>(); // returns null

You can also plug in your own converter in the framework. This allows you to perform very specific type conversions by just calling stringValue.To().

Scheduler

The Scheduler is a collection of classes to allow your application to perform certain tasks at a predefined schedule (for example: every 5 minutes, every monday at 4pm, every 2nd of the month at noon, …)

A few built-in schedulers are included, but you can create your own very easily.

Scheduler scheduler1 = new CyclicScheduler(null, TimeSpan.FromMinutes(30));

// inside your application, call ShouldRun to check if your task should run:
if (scheduler1.ShouldRun) {}

// Monthly or daily schedulers
Scheduler scheduler2 = new MonthlyScheduler("MONTHLY", new TimeSpan(12, 0, 0), 5, 10);
Scheduler scheduler3 = new TimeOfDayScheduler("DAILY", new TimeSpan(12, 0, 0));

Of course, having a scheduler that runs every month or so should have a means of remembering when it ran for the last time. If your application quits, that information will be lost. To handle this, you can attach a HistoryStore object to a scheduler. If you don’t specify one, an in-memory store is used. There is one other built-in store: the FileHistoryStore which stores state in a file on disk.

Scheduler scheduler = new MonthlyScheduler("MONTHLY", new TimeSpan(12, 0, 0), 5, 10);
scheduler.HistoryStore = new FileHistoryStore("c:\\appstate.hst");

There’s more…

Yes, there’s more, but I ran out of breath. I will try to go into more detail in the next few weeks. In the meantime, feel free to download the bits at viciproject.com and check it out.

I’m sure this has been done several times before by hundreds of people, but I wanted to see what I could do with my limited knowledge of Javascript.

The idea was to be able to do something like this:

var Shape = new Class( { color:"white", area: function() { return 0.0; } } );

var Square = Shape.Derive({side:0.0, area:function() { return this.side*this.side; } } );
var Circle = Shape.Derive({r:0.0, area:function() { return this.r*this.r*Math.PI; } } );

var shapes = [];

shapes.push(new Square({side:5.0}));
shapes.push(new Circle({r:5.0}));
shapes.push(new Square({side:3.0 , color:"red"}));

for (var i in shapes)
  console.log("area = " + shapes[i].area() + " , color = " + shapes[i].color);

/*
Should output:
area = 25 , color = white
area = 78.53981633974483 , color = white
area = 9 , color = red
*/

The code required to make this happen is surprisingly simple. A real Javascript ninja could make it even shorter I’m sure:

Class = function(parent, options) {
    if (parent && typeof(parent) != 'function') {
      options = parent;
      parent = undefined;
    }

    var newClass = function(o) { $.extend(this,o); };

    if (typeof(parent) == 'function') {
      newClass.prototype = new parent();
    }

    $.extend(newClass.prototype,options);

    newClass.Derive = function(o) { return new Class(newClass,o); };

    return newClass;
  };

I admit, it does use jQuery ($.extend), but it wouldn’t be hard to write a simple replacement function for that one. It simply copies the properties of a given object to another object. Maybe I’ll add that when I have some time to burn.

As you can see, there are 3 ways to create a class:

var NewClass1 = new Class( prototype );
var NewClass2 = new Class( BaseClass, prototype );
var NewClass3 = BaseClass.Derive( prototype );

In all cases, the prototype parameter is optional. It is used to define functions and default properties for each class.

That’s it. If you think it’s useful, feel free to use it for whatever you like.

• The backend store is in a non-standard format
• It’s not possible to execute ad-hoc SQL queries (neither DDL nor DML)
• Shipping a database file with your app is a complicated process

If any of these limitations is a blocker for your application, or you just want to have a local database right now, using SQLite is a great alternative.

Using CoolStorage for easy SQLite access on Windows Phone

By far the easiest way to use a local SQLite database in your Windows Phone app is by using the free, open-source Vici CoolStorage ORM library. There is a specific native build for Windows Phone which includes a driver for SQLite.

Adding CoolStorage to your app is pretty straightforward:

• Option 1: If you have NuGet installed in VS2010, add the Vici CoolStorage package (available in the NuGet gallery)
• Option 2: If you don’t have NuGet, download the binaries from the Vici Project website and reference both Vici.Core.WP7.dll and Vici.CoolStorage.WP7.dll

Then you add the following line of code somewhere in your initialization code:

    CSConfig.SetDB("mydb.sqlite"); // "mydb.sqlite" is the name of your database file

By default, CoolStorage will create the database file for you if it doesn’t exist. But for this article, we’ll assume you already have a database with data (see here)

CoolStorage is a full-blown ORM with support for relations, lazy loading, relation prefetching, etc, so you will be able to use all of that in your Windows Phone app, but for once I won’t go into that, because if you want you can also use CoolStorage as a lightweight data layer and execute ad-hoc SQL queries.

Here are some examples:

// execute a SQL insert statement
CSDatabase.ExecuteNonQuery("insert into customer (name,city) values (@name,@city)",
                                  new { name="Microsoft", city="Redmond" });

// execute a select SQL statement and map the result to a class

class QueryResult
{
   public string name;
   public int numsales;
   public decimal totalsales;
}

QueryResult[] results = CSDatabase.RunQuery<QueryResult>(
                                        @"select name,count(*),sum(s.total)
                                            from salesperson sp
                                            inner join sales s on s.salespersonid=sp.id
                                            group by sp.name");

// retrieve a scalar value (for example, the total number of customers)

int numCustomers = CSDatabase.GetScalar<int>("select count(*) from customer");

So if you’re not that crazy about ORM’s, or you want to execute some very specific SQL satements, you can do that very easily on Windows Phone with a local SQLite db.

Shipping a SQLite database file with your app

If you have an existing SQLite database file and you want to ship it with your application, you’ll have to make it available in Isolated Storage. This is pretty straightforward, but not always obvious:

First, add your database file to your Visual Studio project and set the build action to “Content” (also leave “Do not copy”).

Then add the following piece of code before calling CSConfig.SetDB(…):

string fn = "mydb.sqlite";

StreamResourceInfo sr = Application.GetResourceStream(new Uri(fn, UriKind.Relative));

IsolatedStorageFile iStorage = IsolatedStorageFile.GetUserStoreForApplication();

if (!iStorage.FileExists(fn))
{
   using (var outputStream = iStorage.OpenFile(fn, FileMode.CreateNew))
   {
      byte[] buffer = new byte[10000];

      for(;;)
      {
         int read = sr.Stream.Read(buffer, 0, buffer.Length);

         if (read <= 0)
             break;

         outputStream.Write(buffer, 0, read);
      }
   }
}

// Now you can use your database

CSConfig.SetDB(fn);

Having the possibility to ship a SQLite database file with your app is especially useful if you want to share a pre-built database with other mobile platforms, since both iPhone and Android have built-in support for SQLite.

In a next post, I will show how to use the ORM features of CoolStorage.

And while you’re at it, you might as well use MonoTouch (and CoolStorage) for building the iPhone version of your app, allowing you to reuse all of your business logic and data layer code.

About a year ago support for MonoTouch was added to the core library, which meant you could use SQLite databases in iPhone and iPad applications when using MonoTouch. Porting CoolStorage to MonoTouch was fairly straightforward because the MonoTouch guys did a great job by building an ADO.NET provider for SQLite in the core MonoTouch library.

Next up: Windows Phone support. I wanted to get this done in time for Microsoft’s Mix conference, but I was shocked to learn there was no ADO.NET or System.Data available. This meant the complete data access layer had to be refactored to use custom interfaces instead of ADO.NET interfaces and classes, which I did. A few days before MIX, CoolStorage was running successfully on Windows Phone using an SQLite databases in isolated storage.

To recap, Vici CoolStorage now supports the following databases out of the box:

• SQL Server (on .NET)
• MySQL (on .NET)
• SQLite (on .NET)
• SQLite (on MonoTouch)
• SQLite (on Windows Phone)

The version supporting Windows Phone (1.5) is not yet released, but a development build can be downloaded from the Vici Project website. The final release of version 1.5 is planned before the end of April.

(*) Rob Conery wrote a very useful and lightweight data layer which he named “Massive“. So I’m obviously not referring to that.

These are some of the formats in use today:

1. {“date”: new Date(ms_since_epoch) }
2. {“date”: Date(ms_since_epoch) }
3. {“date”: “Date(ms_since_epoch) }
4. {“date”: “/Date(ms_since_epoch)/” }
5. {“date”: “\/Date(ms_since_epoch)\/” }
6. {“date”: “yyyy-MM-ddTHH:mm:ssZ” }
7. {“date”: “yyyy-MM-ddTHH:mm:ss” }

Formats 1 and 2 are actually invalid JSON. However, they’re the only formats that were handled correctly by jQuery 1.3.2 and earlier. eval() also handles these. But again, it’s not valid JSON.

That leaves the other 5 formats, which are valid according to the JSON specs.

Starting with version 1.4, jQuery’s built-in JSON evaluator will check if there is a function JSON.parse() available. If it is there, it will use that function to evaluate JSON objects. If not, jQuery will use the “unsafe” eval() way of parsing JSON. JSON.parse() is a function from the json2.js file, which can be downloaded from the JSON website

The problem

JSON.parse() doesn’t handle dates. At all. But it does have a way to handle non-standard values by specifying an extra parameter “reviver”, which is a function that takes a value and returns the value converted to the data type of your choice.

For example: if you want to handle the ISO date format correctly, you can do this:

var parsedObject = JSON.parse(jsonData, function (key, value) {
   var a;

   if (typeof value === 'string') {
     a =/^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2}(?:\.\d*)?)Z$/.exec(value);

     if (a) {
       return new Date(Date.UTC(+a[1], +a[2] - 1, +a[3], +a[4], +a[5], +a[6]));
     }
   }

   return value;
});

(this example is taken from the json2.js source code)

This function will check if a string value parsed from the JSON string is in a specific date format, and return a Javascript date object.

You could expand this function to handle every other possible date format, which shouldn’t be too hard to do, but how do you tell jQuery to use this “reviver” function?

Well, you can’t.

The solution

There’s an easy solution: after including json2.js, redefine the JSON.parse() function so it passes your conversion (reviver) function to the original JSON.parse() function:

<script type="text/javascript" src="json2.js"></script>
<script>
(function() {
   var _origParse = JSON.parse;

   JSON.parse = function(text) {
      return _origParse(text, function(key, value) {
         var a;

         if (typeof value === 'string') {
            a = /^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2}(?:\.\d*)?)Z$/.exec(value);

            if (a)
                 return new Date(Date.UTC(+a[1], +a[2] - 1, +a[3], +a[4], +a[5], +a[6]));

            if (value.slice(0, 5) === 'Date(' && value.slice(-1) === ')') {
               var d = new Date(value.slice(5, -1));

               if (d)
                  return d;
            }
         }

         return value;
      });
   }
})();

</script>

Now when you return some JSON object to your jQuery script, dates will be parsed correctly, without having to change your code. The code snippet above handles cases 3 to 6 correctly. I’ll leave it up to you to add case 7…

But…

When I started developing iPhone apps about a year ago, it felt… awkward. If you want to create an app for the iPhone, you have to use Objective C, which was invented sometime around 1934. I’m kidding of course: it was 1986, but that’s a small detail.

Apart from the odd syntax (which is fine, other languages have odd syntax as well), I was amazed by the lack of basic language and framework features we take for granted these days.

To illustrate the contrast, I’ll show a small function that strips the time portion of a date, written in Objective C:

+ (NSDate *) stripTime:(NSDate *) date {
   NSCalendar *gregorian =
         [[NSCalendar alloc] initWithCalendarIdentifier:NSGregorianCalendar];

   NSDateComponents *components =
          [gregorian components:(NSYearCalendarUnit | NSMonthCalendarUnit | NSDayCalendarUnit)
                       fromDate:date];

   date = [gregorian dateFromComponents:components];

   [gregorian release];

   return date;
}

The same thing in C#:

public DateTime StripTime(DateTime date)
{
    return date.Date;
}

I guess I have been spoiled by C#. Can you imagine what it would be like to consume a SOAP webservice from Objective C? Well, check this out. (I have to admit: it received a 5-star rating, so I guess it was worth the 62 hours spent on it)

PS. I realize I suck at Objective C, and I’m sure a better Objective C developer can shave off a line or 2 of my code, but you get the picture…

Thank god a few geniuses over at Novell created MonoTouch…

Being a C# .NET programmer didn’t make this easy: Objective-C is… weird and… primitive. Dealing with pointers after 10 years of focusing on problems instead of memory addresses is a slight culture shock, but luckily having previous C and C++ experience helps a lot.

In September last year, the geniuses (by lack of a stronger word) at Novell’s Mono team came up with MonoTouch, a C# cross-compiler that allows you to build iPhone apps using .NET and C#, including the use of most of the .NET Framework. I’m not going to go into detail on how this works. Enough has been written about this subject.

One feature of MonoTouch caught my attention: as of version 1.2, an ADO.NET provider was added for accessing the iPhone’s native SQLite database format. A relational database accessed from C# code desperately needs an ORM, don’t you agree? So last weekend I decided to try and port the CoolStorage ORM to MonoTouch. It wasn’t as easy as I thought, because of some severe limitations in the version of SQLite installed on the iPhone, but in the end I got it working, and working well.

To give you an idea of the time you’ll save by using an ORM like CoolStorage on the iPhone, I will present three examples:

1. Reading a list of records using the iPhones’s SQLite library (C / Objective-C)
2. Reading a list of records using ADO.NET + MonoTouch
3. Reading a list of records using CoolStorage + MonoTouch

We’re not going to make it too complicated. The example reads a list of customer records. It retrieves the ID, Name and City. No filters are being applied, simply a list of customers ordered by Name. After that, it prints the records to the console.

The examples also omits the declaration of the Customer class (which is used to hold a single customer record)

Native Objective-C example:

NSArray *documentPaths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *documentsDir = [documentPaths objectAtIndex:0];

databasePath = [documentsDir stringByAppendingPathComponent:@"mydb.db3"];

sqlite3 *database;

customers = [[NSMutableArray alloc] init];

if(sqlite3_open([databasePath UTF8String], &database) == SQLITE_OK) {
   const char *sql = "select id,name,city from customer order by name";
   sqlite3_stmt *stmt;

   if(sqlite3_prepare_v2(database, sql, -1, &stmt, NULL) == SQLITE_OK) {

      while(sqlite3_step(stmt) == SQLITE_ROW) {
           Customer *customer = [[Customer alloc] init];

           customer.CustomerID = sqlite3_column_int(stmt, 0)];
           customer.Name = [NSString stringWithUTF8String:(char *)sqlite3_column_text(stmt, 1)];
           customer.City = [NSString stringWithUTF8String:(char *)sqlite3_column_text(stmt, 2)];

           [customers addObject:customer];

	   [customer release];
      }
   }

   sqlite3_finalize(stmt);
}

sqlite3_close(database);

for (int i=0;i<[customers count];i++) {
   Customer *customer = [customers objectAtIndex:i];

   NSLog("ID: %d, Name: %@, City: %@",customer.CustomerID,customer.Name,customer.City);
}

It’s pretty obvious that things will get unbearably complicated when you try to do things like accessing related records, persisting data to the database, and so on.

Now let’s introduce MonoTouch and ADO.NET.

ADO.NET example (MonoTouch):

string dbName = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Personal), "mydb.db3");

var conn = new SqliteConnection("Data Source=" + dbName);

List customers = new List();

using (var cmd = conn.CreateCommand())
{
  conn.Open();
  cmd.CommandText ="select id,name,city from customer order by name";
  using(var reader = cmd.ExecuteReader())
  {
    while (reader.Read())
    {
      Customer customer = new Customer();

      customer.CustomerID = int.Parse(reader["id"].ToString());
      customer.Name = (string) reader["name"];
      customer.City = (string) reader["city"];

      customers.Add(customer);
    }
  }
}

foreach (var customer in customers)
   Console.WriteLine("ID: {0}, Name: {1}, City: {2}",customer.CustomerID,customer.Name,customer.City);

Now that’s a lot better, don’t you think? But still, it requires a lot of boilerplate code that’s just there to distract you from what you’re actually trying to do: read a list of records from the database.

So let’s move to the next level:

CoolStorage example (MonoTouch):

string dbName = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Personal), "mydb.db3");

CSConfig.SetDB(dbName, false);

var customers = Customer.List().OrderedBy("Name");

foreach (var customer in customers)
   Console.WriteLine("ID: {0}, Name: {1}, City: {2}",customer.CustomerID,customer.Name,customer.City);

Well, you get the picture… Now you can focus on the application instead of data access technology. Isn’t this what it’s all about?

In case you want to use this in your own code, go ahead and grab the latest development build of CoolStorage, which now fully supports MonoTouch. It can be downloaded from the CoolStorage website.

Best of all, it’s open source…

What is equality?

Equality means different things in different programming languages. In most modern languages (C# for example), you can define your own terms of what equality means. In others you can’t.

Take javascript for example. There are actually two kinds of equality: normal equality and being identical.

To check if two variables or values are equal, you use:

f (a == b) {
  // ...
}

To check if two variables are identical, you use:

if (a === b) {
  // ...
}

Most articles (and even supposedly good books) define the latter comparison as being equal and of the same type. While this over-simplification may be true in the majority of cases, it is very inaccurate, because you have to know what “equal” means. I’m not going to talk about javascript’s normal equality operator (==), because enough has been written about that. I’ll focus on the === operator.

The correct meaning of === is that the 2 operands should be identical. This means they should reference the same object, or, in the case of value types, the values should be the same. Only numbers and booleans are value types. Strings behave as value types, but they are actually reference types.

Bringing this all together, let’s look at some examples of equality in javascript:

var a = 1;
var b = 1;

alert(a == b); // true
alert(a === b); // true

b = "1";

alert(a == b); // true
alert(a === b); // false

These are the obvious ones. Now the more interesting stuff:

var a = [1,2,3];
var b = [1,2,3];
var c = a;

alert(a === b); // false (these look equal to me, and of the same type)
alert(a === c); // true

Remember I mentioned that strings behave like value types? Now it becomes interesting:

var a = "12" + "3";
var b = "123";

alert(a === b); // returns true, because strings behave like value types

And to make it really interesting:

var a = new String("123");
var b = "123";

alert(a === b); // returns false !! (but they are equal and of the same type)

I thought strings behave like value types? Well, it depends who you ask…

EDIT: As one of the commenters (zihotki) points out, the last example adds another twist to the story. Creating a string using the String() object constructor actually doesn’t create a variable of type “string”, but of type “object”. But you can use the object as a string. If you are unaware of this behavior, you can easily shoot yourself in the foot without knowing it.

A few months ago, just before the release of ProMesh.NET version 2.0, we decided to move the project to the Vici Project, a project that bundles several open-source frameworks for .NET 2.0. The idea of the Vici Project is to provide .NET developers with a collection of lightweight libraries and frameworks, and at the same time get the community involved in the development and support of these libraries.

To make it easier to get the community involved, a complete system was created with the following features:

• Central SubVersion repository with online browsing (using WebSVN)
• Automated build server (using JetBrains TeamCity)
• Wiki infrastructure for maintaining documentation
• Support forum

After several months of testing all of this, the project is finally ready to go live. There is still a lot of work to be done, especially on documentation, but I think there is no point in postponing the release.

The first sub-project of the Vici Project to be released is Vici MVC, formerly known as ProMesh.NET. Later this week, 2 other projects will be released as well: Vici Parser (formerly LazyParser.NET/SharpTemplate.NET) and Vici CoolStorage (formerly CoolStorage.NET)

Vici MVC 2.0 new features (compared to ProMesh.NET 1.2)

• New powerful URL routing engine. Also supports “extension-less” URLs with IIS 7.0 or IIS 6.0 (with wildcard mapping or URL rewriting)
• Support for view components (“inline” controllers with templated views)
• Support for sub-templates with parameters
• Support for template macros
• Configurable template syntax
• Full C# 2.0 expression supported in templates
• New template language syntax (the old one is still supported)

Anyone interested in contributing to the Vici Project, please let me know by posting on the forum (or by sending me a private message on the forum. My username is activa)

Early next year, a cool new service will go online targeted at web developers and designers. It will go by the name of dStyler.com but it’s a little hard to explain what the service will do, but I’ll start with a simple example:

What dStyler actually provides is real-time online image generation. It is a highly scalable REST-style webservice that generates images based on a custom-built URL. For example, the box on the right is generated by the following url:

http://dstyler.com/ufnqfr/204×254/bo-333/ib-fff-2/ro-5/gr-eef-b8e/gr-fff9-fff0-210×260/sh-0008-0-r3.jpg.

It is in a way similar to the Google Charts API and the Google static maps API.

Some of the things dStyler can generate:

• Solid backgrounds
• Gradients
• Single or double borders
• Drop shadows
• Image embedding (useful for watermarks)
• Special effects
• Rounded corners
• Slicing
• …

The online administration interface allows you to:

• Build images in an interactive way (to generate the URL for you)
• Upload images to your account that can be used for generating images

Best of all, dStyler.com will be a free service

We are currently looking for people interested in beta testing this service. So if you’re interested, contact me at beta -at- dstyler -dot- com (I know, I hate these obfuscated e-mail addresses more than anyone, but sadly enough, the world is a nasty world filled with f**king spammers)

But what if you need to test code that relies on the date or some elapsed time? Let’s say you wrote a simple caching class that automatically removes items from the cache when a certain amount of time has passed since the last access. It’s tempting to slap some Thread.Sleep() calls in there to trigger expiration of cache items, but that will slow down your unit test, and most importantly, it’s not accurate. What if you want to test some edge cases? Like, what happens when you access an item at the exact timeout period?

A few years ago I worked with a brilliant Java architect who simply said: “just mock the time!”. What he meant was that I should create a fake DateTime class which behaves like the real DateTime class with one important difference: YOU control the time, not the system clock.

How is that done?

Simply create an interface with a single getter property of type DateTime:

public interface ITimeProvider
{
   DateTime Now { get; }
}

Then you create 2 classes that implement this interface: one that returns the real time, and one that can be used to control a “fake” time:

public class RealTimeProvider : ITimeProvider
{
    public DateTime Now { get { return DateTime.Now; } }
}

public class MockTimeProvider : ITimeProvider
{
    private DateTime _time;

    public DateTime Now { get { return _time; } set { _time = value; } }
}

Of course the class(es) you want to test should be aware of this. For example, our cache class could look like this:

public class Cache
{
     private ITimeProvider _time = new RealTimeProvider();

     public ITimeProvider TimeProvider { get { return _time; } set { _time = value; } }

     // ... the rest of the class implementation
}

In our class implementation, all calls to DateTime.Now should be replaced by TimeProvider.Now.

And now the fun part, faking the time in our unit tests:

[Test]
public void TestCacheTimeout()
{
    ITimeProvider time = new MockTimeProvider();
    Cache cache = new Cache();

    cache.TimeProvider = time; // Tell our cache class to use our fake time

    cache.Add("A" , 1); // add an item to the cache

    Assert.IsTrue(cache.Contains("A")); // check if it's added

    time.Now += TimeSpan.FromHours(1); // let one hour go by...

    Assert.IsFalse(cache.Contains("A")); // check if it was automatically removed
}

That’s pretty cool and simple, isn’t it? Wouldn’t it be nice if we could do the same in real life?

Next week I’ll talk a little about unit testing multithreaded concurrency issues…

Anyway, the thing is, as Jeff points out, URL’s can contain some pretty weird characters you wouldn’t expect, like ‘(‘, ‘)’ and ‘,’. This can become quite frustrating when trying to parse something like this:

My website (at http://www.mysite.com/coolpage) is becoming very popular, especially because of traffic coming from http://www.othersite.com/coolestpage, which is a cool site I recently found.

For humans, it’s pretty obvious where the hyperlinks are, but what if the links contain some of the non-standard characters I mentioned above? For example:

http://www.mysite.com/coolpage(nice).aspx or http://www.othersite.com/coolest,wildestpage. Both of these are valid URLs.

If you would implement a trivial URL detector, the following URLs would be extracted from the text snippet above:

“http://www.mysite.com/coolpage)” and “http://www.othersite.com/coolestpage,“.

Note the trailing parenthesis and comma. Obviously that is not what we want. The fact is that there is no way we can extract valid URLs by following a set of fixed rules, not even for humans. We humans will “parse” the URLs by looking at the context, but that’s a pretty hard, if not impossible task for software.
What we can do is use some heuristics to satisfy at least 99.99% of all cases, so I started thinking about this a little bit and came up with a solution that uses regular expressions and requires no code at all.

Let’s start with the basics. A URL contains 3 distinct parts, of which the last part is optional:

protocol://host/path

“host” is a hostname (or IP address), optionally followed by a port number, so this is the regex we can use for this:

([a-zA-Z-:@.0-9]+)

Of course, this will not validate the host name part of the URL, but that’s not really our goal now.

Next we should specify what the path looks like. According to the specs, the path can contain any of these characters: letters, digits and any of -;:@&=$_.+!*’,()

In this example, we will only try to match http and https, so our trivial regex would look like:

\b(https|http)://([a-zA-Z-:@.0-9]+)(/[-;:@&=?a-zA-Z0-9$_.+!*',()])?

That will work in most cases, but it fails miserably in our text snippet example.

Let’s make an assumption: we assume that a URL containing parentheses always has matching parentheses, as in “http://www.mysite.com/Some(Nice)Page“. This would exclude URLs with a single parenthesis, so “http://www.mysite.com/Some(NicePage” would not be recognized as a URL. That’s a sacrifice I’m willing to make.

There are several ways of writing a regex like this, but a simple one is:

\b(https|http)://([a-zA-Z-:@.0-9]+)(/((\([-;:@&=a-zA-Z0-9$_.+!*',]*?\))|[-;:@&=?a-zA-Z0-9$_.+!*',]|%\d\d)+)?

Note that I also added some regex code for escaped characters (%nn). Nested parentheses will not be matched.

This would solve the first match in our text snippet, but not the second one with the trailing ‘,’. Again, we will make an assumption: let’s assume that a URL will never end with a period or a comma. I think that’s a pretty safe assumption to make (although technically it is allowed).

Our final regex will then look like:

\b(https|http)://([a-zA-Z-:@.0-9]+)(/((\([-;:@&=a-zA-Z0-9$_.+!*',]*?\))|[-;:@&=?a-zA-Z0-9$_.+!*',]|%\d\d)+)?(?<![,.;])

This regex will match any valid URL pattern, except URLs without matching parentheses. It will also exclude URLs with a period, comma or semicolon at the end.

Again, it’s not a perfect solution, but I bet it will be very hard to find a real-world example where this regex would fail to extract the URL from a piece of text. The regex can still use some tweaking, but you get the picture.

The only problem is that when you use your own software, you always have the feeling that it’s not quite “done”. There’s always something that you can do better, always something that can be improved, always some features that you feel would be nice.

Especially when building a framework this can become a real problem because you never feel done. Another problem is making sure your framework maintains backwards compatibility with previous releases.

Well, I just mentioned this because I finally decided to freeze version 2.0 of ProMesh.NET and get it ready for release. Release candidate 3 has just been published on CodePlex

Thoughts on open source frameworks

A few weeks ago, a fellow developer asked me why I chose to make my projects open-source. That was a pretty good question because when you look at it: compared to just building a framework for your own use, it makes life more complicated: you have to write documentation, support users, and more headaches.

I didn’t have to think long before I could answer that question: releasing a custom framework as open source forces you to be disciplined about your code and documentation. After all, you can’t afford to write code you are ashamed of, can you? If you write software for your own use, you tend to write code that… well… sucks.

And of course, let’s not forget the whole point of open source software: letting other developers contribute. It creates a dynamic that you would never get when you’re just building and using your own little framework

If all of the above sounds like a bunch of crap, it’s probably because I’m not a native English speaker, or … it is just crap

It took a while to figure out what was going on. It seems the weird bytes were the byte order mark for UTF-8 documents (hex EF BB BF). When Google Chrome finds these bytes within a <script> block, it will simply stop parsing javascript. No errors, no warnings.

You can argue that this is a problem with Google Chrome, but that still doesn’t explain why these bytes were in my javascript blocks to start with.

The problem is this: I inject javascript in the HTML output from embedded resources. The embedded resources are just .js files created in Visual Studio, marked as “embedded resource”. The resource is then read from the assembly and converted to a UTF8 string and inserted in the HTML script block. The problem is that Visual Studio ALWAYS adds the UTF-8 “byte order mark” when you save a text file. These bytes are also embedded in the embedded resource, which is… annoying.

Of course, you could tell Visual Studio to save your file without signature (“byte order mark”), but you have to do that EVERY time you’ve created a new javascript file. There’s no way to make that the default.

Bummer…

On a positive note, this gives me the perfect excuse to announce the release of Release Candidate 2 of ProMesh.NET 2.0, which has built-in detection for BOM bytes on embedded resources

No, not writing the code.

No, not helping users in the support forum…

I’ll tell you: writing documentation, especially when the documentation is not in your native language. So that’s what I have been doing for the last few weeks (or should I say “months”?).

In fact, the documentation project was the only thing that held back the release of ProMesh.NET version 2.0, and today I am proud to announce the release of the first release candidate of ProMesh.NET 2.0, with full documentation online. The documentation still needs a lot work, but at least (almost) all the features of the framework have been documented.

So, what’s new in ProMesh.NET 2.0 (RC1) ?

• Built-in routing engine. Routing can be added at runtime (at application startup), or can be specified using attributes on the controller classes and action methods.
• Support for calling/rendering view components from templates. View components are special controllers that render a view, which will be inserted in the calling view.
• Support for any page extension, including none at all (recommended for IIS 7 deployment)
• New template engine (SharpTemplate.NET) which supports full C# expressions in view logic
• Ajax validation of forms
• Performance improvements

Want to check it out? Why don’t you start where all the new stuff is: The Official ProMesh.NET Website

Or if you want to go straight to the download: Go to CodePlex

1. 3G (HSDPA)
2. Support for push e-mail (MS Exchange Server)
3. Web browsing (even if it’s flaky)
4. Small form factor

Looking at the list, I may not be that picky after all. Feature number 2 however forces me to use Windows Mobile, and to be honest, I never think twice about that when deciding on a smartphone purchase. So these were my latest phones (in order of purchase)

• HTC MteoR (WM 5.0)
• HTC S730 (WM 6.0)
• HTC Touch Diamond (WM 6.1) – my current phone

What do these have in common? Of course, they’re all HTC, but most importantly they’re all Windows Mobile devices.

You would think progress means that every generation of a technology improves on features, power and responsiveness, but once you mix in products made by some huge company from Redmond, this universal law no longer applies.

Of the smartphones listed above, the fastest one was the MteoR, followed by the S730 and … well, I almost threw the damn thing into a concrete wall several times since I bought the Diamond 2 months ago.

What the hell is wrong with Windows Mobile? Why, after so many years of development, after so many releases can’t they get their act together and create a user interface that does something within, like, 2 seconds of acting upon it. Tapping something on the screen has become an act of blind faith because there’s no immediate feedback. You just hope something is going to happen, and when you’re in a hurry (you always are, otherwise you wouldn’t need a smartphone), you tap again, only to find out that the “system” was already processing your first tap and sending your second tap to the button or link that will be in that same spot on the next screen within 5 seconds or so. If you’re lucky, that button will delete an important file or hang up on an important client. Thank you so much, mighty Microsoft.

How does Microsoft expect to win the battle against Apple or Google (Android) in the smartphone world? If they even can’t get the basic stuff right. Don’t tell me it can’t be done on small devices. Apple did it, and from what I’ve seen so far, Google did it.

One small piece of advice for the Windows Mobile developers: focus on the immediate feedback of the user interface, NOT on features. Rewrite all the user interface handling from scratch, put the user interface code in a dedicated area of memory that can’t be “virtualized” (is there such a thing in Windows Mobile?) and get your act together.

My iPhone 3G will arrive tomorrow. Amen. (thank you Apple for licensing MS Exchange for the iPhone 3G)

• The framework has a lot of new features, like a built-in routing engine similar to ASP.NET MVC and support for view components
• Some small changes are required to existing application using ProMesh.NET v1.2

At the same time of the release (late August, early September), the full documentation will be available on a dedicated website about ProMesh.NET.

A quick overview of the most important features in version 2.0:

• URL routing engine similar to ASP.NET MVC
• Support for view components (combination of code+templates callable from templates)
• Configurable template language syntax
• Full C# expression parser available in the template language
• New template language syntax (but the old one is still supported)
• Support for extension-less URLs

Watch this space for the release announcement in a few weeks!

Besides a bunch of bug fixes and performance improvements, the new features in this release are:

• Support for prefetching of many-to-one, one-to-one and one-to-many relations (marked by the [Prefetch] attribute at compile-time or by using the .WithPrefetch() method). This change results in a spectacular performance improvement in most cases where lists of objects with related objects are retrieved from the database.
• Support for paging at server level. Use the Range() method on the CSList class to get a specific range of records
• Support for GUID fields (server-generated and client-generated)
• Support for specifying a context when calling generic CSDatabase.RunQuery() methods
• New method ToList<T> on CSList
• New method ThenBy() on CSList (in line with the LINQ methods)
• New attribute [ClientGenerated] for GUID fields
• New attribute [NotMapped] for fields that are not mapped to a database column
• Support for VistaDB 3.x database

Databases currently supported are:

• MS SQL Server 2000/2005
• MS Access
• SQLite v3
• VistaDB 3.x
• MySql 5.x
• IBM DB2

The latest release can be downloaded from CodePlex.

In case you have no idea what CoolStorage.NET is:

CoolStorage.NET is a fully typed Object Relational Mapping library for .NET 2.0.

The main strength of CoolStorage.NET is the ease of use. Most ORM tools still require a lot of type casting and other plumbing to be written, CoolStorage.NET is designed to relieve the programmer from these tedious and error-prone tasks, making it very intuitive to use. It is “session-less”, meaning you don’t have to keep a connection or session object lying around to access objects in the database.

The main features are:

• Supports SQL Server 2000/2005, MySQL, IBM DB2, SQLite, MS Access, VistaDB
• Any existing relational data model can be mapped to CoolStorage objects with minimal effort
• All relation types are supported: One-To-One, One-To-Many, Many-To-One and Many-To-Many
• Completely typed object model (no type casts required)
• Full support for transactions, including .NET 2.0 TransactionScope
• Nullable columns can be mapped to .NET 2.0 nullable fields or any other value
• Delayed (lazy) loading of data to minimize database access
• Selective pre-fetching of all relation types for improved query performance.
• Powerful and intuitive database-independent object query language
• Flexible event framework to intercept any event
• Identity (auto-increment) keys are supported for all database types
• Support for server and client generated Guid keys
• Sessionless data access
• Objects can be mapped to different databases, even across object relations
• Pageable object collections
• Extensive support for retrieving aggregate values on collections (count, sum, average, …)
• Collections implement IBindingList so they can be used by controls (grids) as a data source
• Underlying database engine uses optimized and parameterized SQL queries. SQL injection is impossible
• Built specifically for .NET 2.0, taking full advantage of generics and nullable variables
• Raw SQL and/or stored procedures can be called on the underlying database without the need for a separate database connection
• Small footprint (less than 120 KB)

Personally, when I’m using a nice, clean font for my coding, I feel happy about my code. The code just looks better. And I don’t mean it “looks” better, but it looks better (catch my drift?)

I have tried a lot of fonts, and for the last few years I’ve always gone back to Damien Guard‘s excellent Envy Code R font. It is (IMHO) the perfect font for source code.

Today Damien released “Preview #7″ of his font, which is a big step forward: the font looks almost perfect at all sizes. I really recommend it to anyone. And it’s free! (he does accept donations, which he really deserves)

The link to his announcement: Envy Code R Preview #7 released

The .NET documentation tries to sum it up in one sentence:

A data type is a value type if it holds the data within its own memory allocation. A reference type contains a pointer to another memory location that holds the data.

This means several things:

• You cannot set a variable of a value type to null. It’s not a pointer. It’s just there.
• When returning a value type from a method, a copy is made and the whole value is returned, not just a reference.
• Creating a variable always initializes it to “something”.

All primitive numeric types are value types (Int32, Double, Int64, Boolean, …), as well as DateTime, enums and structs. Examples of reference types are String, all arrays (even when they contain value types) and all classes.

So far nothing new, but to make it interesting, let’s create a small sample application using a custom struct, which is a value type:

Struct declaration:

struct MyStruct
{
  public int Value = 0;

  public void Update(int i) { Value = i; }
}

Code sample:

MyStruct[] list = new MyStruct[5];

for (int i=0;i<5;i++)
  Console.Write(list[i].Value + " ");

Console.WriteLine();

for (int i=0;i<5;i++)
  list[i].Update(i+1);

for (int i=0;i<5;i++)
  Console.Write(list[i].Value + " ");

Console.WriteLine();

The output of this code is:

0 0 0 0 0
1 2 3 4 5

Now let’s do the same, but substitute the array for a generic List<>:

List<MyStruct> list = new List<MyStruct>(new MyStruct[5]); 

for (int i=0;i&lt;5;i++)
  Console.Write(list[i].Value + " ");

Console.WriteLine();

for (int i=0;i&lt;5;i++)
  list[i].Update(i+1);

for (int i=0;i&lt;5;i++)
  Console.Write(list[i].Value + " ");

Console.WriteLine();

The output is:

0 0 0 0 0
0 0 0 0 0

Surprised? I was too, at first, but the explanation is very simple. No, it’s not boxing/unboxing…

When accessing elements from an array, the runtime will get the array elements directly, so the Update() method works on the array item itself. This means that the structs itself in the array are updated.

In the second example, we used a generic List<>. What happens when we access a specific element? Well, the indexer property is called, which is a method. As I mentioned earlier, value types are copied when they are returned by a method, so this is exactly what happens: the list’s indexer method retrieves the struct from an internal array and returns it to the caller. Because it concerns a value type, a copy will be made, and the Update() method will be called on the copy, which of course has no effect on the list’s original items.

What should we learn from this?

Always make sure your structs are immutable, because you are never sure when a copy will be made. Most of the time it is obvious, but in some cases it can really surprise you…