activa's blog

The Vici Project has been around for about 4 years now, and although most of the attention has gone to CoolStorage, the unsung hero of the framework collection is Vici Core, a general-purpose toolkit for .NET and Mono.

I bet most people don’t have a clue what Vici Core is, and the current website doesn’t offer a lot of insight either, so I thought it might be a good idea to give an overview of the components included in the toolkit. I will follow up with additional blog posts describing the components in more detail.

What is Vici Core?

The most important parts of Vici Core are the dynamic expression parser and template rendering engine. These components allow the developer to incorporate runtime expression parsing and template rendering in their applications. The parser is generic, meaning that it can handle any imaginable syntax, but it includes a C# style expression parser in the core library.
The parser and template renderer are both based on a generic and extensible tokenizer. The tokenizer is also used by the built-in JSON parser.

This is a quick overview of the components in Vici Core:

• Tokenizer
• Expression Parser (C# built-in)
• Template engine
  • Mustache syntax
  • Velocity syntax
  • Vici MVC syntax
• JSON parser
• JSON serializer
• Configuration
• Logging
• Exception-free smart type conversion
• SmartCache
• FastReflection
• Notifications/Subscriptions
• Scheduling
• Binary and Base64 serialization

Best of all, these components can be used on the following platforms:

• .NET 3.5 or higher
• Windows Phone 7.5
• MonoTouch
• Mono for Android

If you want to check it out, feel free to download the source and/or binaries at viciproject.com

Below is a description of most of the components in the framework.

Tokenizer

The Vici Core tokenizer allows you to tokenize any stream of characters to a list of tokens. The tokenizer is extensible so you can define your own tokenizer logic by creating classes implementing the ITokenParser and ITokenProcessor interfaces.

Using the tokenizer, you can take the following string:

Items["Key"] = 45;

And turn it in the following list of tokens:

The code for this is would be:

Token[] tokens = new CSharpTokenizer().Tokenize("Items[\"Key\"] = 45;");

To give you an idea of the way you would write your own token recognizer, this is the source code of one of the built-in classes that will recognize a string (keyword):

    // This class does double duty, implementing both ITokenMatcher and ITokenProcessor
    public class KeywordMatcher : ITokenMatcher, ITokenProcessor
    {
        private int _index;
        private readonly string _string;

        public KeywordMatcher(string s) { _string = s; }

        public ITokenProcessor CreateTokenProcessor()
        {
            return new KeywordMatcher(_string);
        }

        public void ResetState() { _index = 0; }

        public TokenizerState ProcessChar(char c, string fullExpression, int currentIndex)
        {
            if (_index >= _string.Length)
                return TokenizerState.Success;

            return (c == _string[_index++]) ? TokenizerState.Valid : TokenizerState.Fail;
        }

        public string TranslateToken(string originalToken, ITokenProcessor tokenProcessor)
        {
            return originalToken;
        }
    }

Expression Parser

Vici’s expression parser builds an expression tree from any expression. Internally, it tokenizes the input string, reorders the tokens to RPN notation (using the Shunting Yard algorithm) and then creates a true expression tree that can be used to evaluate the expression.

Although you could use Vici’s parser to parse any syntax, only the C# syntax is supported in the core library.

In its simplest form, this is how you could use the expression parser:

CSharpParser parser = new CSharpParser();

int value = parser.Evaluate<int>("5+4*a",new CSharpContext(new {a=10}));

// value == 45

Since it is a true C# expression parser, you can use any C# (2.0) feature you like:

CSharpContext context = new CSharpContext();
CSharpParser parser = new CSharpParser();

context.AddType("Math",typeof(Math));
context.Set("f", new Func<int,int>(i => i*2));
context.Set("obj", new { Value=5 });

int value1 = parser.Evaluate<int>("5 * f(2)", context);
double value2 = parser.Evaluate<double>("Math.Cos(1.0)", context);
string value3 = parser.Evaluate<string>("obj.Value.ToString()", context);

Template Engine

Vici’s template rendering engine is based on the built-in expression parser, adding support for extra control structures like conditional statements, loops and external references.

The templates can be anything from simple strings to HTML files and mail templates. As an example, I’ll present a real-world mail template that could be used to send out order confirmation e-mails to a customer:

Dear $Order.Customer.Prefix $Order.Customer.LastName,

Thank you for your order. The estimated shipping date is $Order.EstShipDate

Your order:
#foreach(item in Order.OrderItems)
$item.Description ($item.Price)
#end

Total price: $ $Order.TotalPrice
...

Rendering this template is done in just a few lines of code:

Order order = OrderService.ReadOrder(...);

CSharpContext data = new CSharpContext(new { Order = order });
VelocityTemplateParser parser = new VelocityTemplateParser();

string renderedMail = parser.Render(template, data);

MailService.SendMail(order.Customer.Email, renderedMail);

This is just one of the many ways you could use the template engine. In addition, it’s very easy to create your own template syntax definitions. Check out the source code of the built-in template syntax handlers to learn more.

JSON Parser

JSON data can be parsed to a dictionary or to a real object. The parser will try to match the JSON data structure to the target object’s properties or fields.

// json:
// { "firstname" : "John" , "lastname" : "Doe", "children" : [ "Kelly","Tim" ] }

public class Person
{
    public string FirstName;
    public string LastName;
    public string[] Children;
}

// ...
JSONParser parser = new JSONParser();

Person person = parser.Parse<Person>(json);

Configuration Framework

The configuration framework allows you to map data from a data source (XML file, app.config, database, …) to class properties and fields.

For example, if this is your app.config file:

<appSettings>
    <add key="Database.Server" value="db.mycompany.com" />
    <add key="Database.Port" value="6005" />
    <add key="Database.Schema" value="Customers" />
    <add key="LogFile" value="Z:\Logs\Logfile.txt" />
</appSettings>

Then you can easily map this to some static “Config” object in your application:

public class Config
{
     public class _Database
     {
          public string Server;
          public int Port;
          public string Schema;
     }

     public static _Database Database;
     public static string Logfile;
}

//...

ConfigManager configManager = new ConfigManager();

configManager.Register<Config>();

configManager.RegisterProvider(new ConfigurationProviderAppConfig());
configManager.Update();

//...

Database.Connect(Config.Database.Name, Config.Database.Port);

The source of your configuration can be anything you like. The framework will call your configuration provider class and the provider simply has to return key/value pairs. Keys can be structured using dotted notation. The values will be converted to the target datatype (strings, numbers, arrays, …)

Versioning is also supported. Your configuration objects will be updated when a version number is changed in your source file.

Logging

Logging is something that can be found in a lot of third-party libraries, but since it’s not that complicated to implement, it was added to Vici Core to minimize dependencies. The end result is a pretty decent logging framework.

It features:

• Pluggable logging providers (logging to multiple providers at the same time)
• Log levels (Debug,Warning,Error,Fatal, …)
• Log rotation and cleanup (by hour,day,week,month,…)
• Dynamic log file names (with embedded dates)

In its simplest form, logging can be added to your application like this:

Logger.Default.AddProvider(new LogProviderFile("c:\\logs\\logfile.txt"));

//...
Logger.Default.Log(LogLevel.Debug, "User logged in: {0}", userName);
//...
Logger.Default.LogException(ex);
//...

SmartCache

What’s in a name? SmartCache is a very simple but flexible caching framework that supports most basic features a caching solution should have:

• Configurable cache size
• Fully typed
• Thread-safe
• Cache entries can be configured to time out after a specific time or at a specific absolute time

Code sample:

var cache = new SmartCache<Record>(500); // max. 500 cache entries

// add item, does not expire
cache.Add(record1.Key, record1);
// add item, expires after 30 minutes when not accessed
cache.Add(record2.Key, record2, TimeSpan.FromMinutes(30));
// add item, expires in exactly 10 minutes
cache.Add(record3.Key, record3, DateTime.Now.AddMinutes(10));

// Retrieving:

// Checks if a record is cached and retrieves it
if (cache.TryGetValue("key1", out record)) ...

// Tries to retrieve a record from the cache, and if not found
// the supplied delegate will be called to fetch the record again
// and store it back in the cache
cache.TryGetValue("key2", out record, () => Database.LoadRecord("key2"));

Type Conversion

The Smart Conversion library allows you to convert any datatype to another data type, without risking type conversion exceptions.

Heuristics are used to convert values from one type to another.

Examples:

string s1 = "25";
string s2 = "0";
string s3 = "";
string s4 = null;
string s5 = "a";
enum LogLevel { Debug = 0, Information = 1, Error = 2, FatalError = 3 };

int a = s1.To<int>(); // returns 25
int b = s2.To<int>(); // returns 0
int c = s3.To<int>(); // returns 0
int d = s4.To<int>(); // returns 0
int e = s5.To<int>(); // returns 0
int? e = s1.To<int?>(); // returns 25
int? f = s2.To<int?>(); // returns 0
int? g = s3.To<int?>(); // returns null
int? h = s4.To<int?>(); // returns null
int? i = s5.To<int?>(); // returns null

LogLevel ll1 = "1".Convert<LogLevel>(); // returns LogLevel.Information
LogLevel ll2 = "3".Convert<LogLevel>(); // returns LogLevel.FatalError
LogLevel ll3 = "Error".Convert<LogLevel>(); // return LogLevel.Error

// If a value can't be converted, the default is returned, which is
// the value 0 (zero) casted to the enum type (according to the C# specs).
// If the target type is nullable and the conversion fails, null is returned.

LogLevel ll4 = "5".Convert<LogLevel>(); // returns LogLevel.Debug
LogLevel ll5 = "Bogus".Convert<LogLevel>(); // returns LogLevel.Debug
LogLevel? ll6 = "5".Convert<LogLevel?>(); // returns null
LogLevel? ll7 = "Bogus".Convert<LogLevel?>(); // returns null

You can also plug in your own converter in the framework. This allows you to perform very specific type conversions by just calling stringValue.To().

Scheduler

The Scheduler is a collection of classes to allow your application to perform certain tasks at a predefined schedule (for example: every 5 minutes, every monday at 4pm, every 2nd of the month at noon, …)

A few built-in schedulers are included, but you can create your own very easily.

Scheduler scheduler1 = new CyclicScheduler(null, TimeSpan.FromMinutes(30));

// inside your application, call ShouldRun to check if your task should run:
if (scheduler1.ShouldRun) {}

// Monthly or daily schedulers
Scheduler scheduler2 = new MonthlyScheduler("MONTHLY", new TimeSpan(12, 0, 0), 5, 10);
Scheduler scheduler3 = new TimeOfDayScheduler("DAILY", new TimeSpan(12, 0, 0));

Of course, having a scheduler that runs every month or so should have a means of remembering when it ran for the last time. If your application quits, that information will be lost. To handle this, you can attach a HistoryStore object to a scheduler. If you don’t specify one, an in-memory store is used. There is one other built-in store: the FileHistoryStore which stores state in a file on disk.

Scheduler scheduler = new MonthlyScheduler("MONTHLY", new TimeSpan(12, 0, 0), 5, 10);
scheduler.HistoryStore = new FileHistoryStore("c:\\appstate.hst");

There’s more…

Yes, there’s more, but I ran out of breath. I will try to go into more detail in the next few weeks. In the meantime, feel free to download the bits at viciproject.com and check it out.