Introducing PartitionCache. A C# cache for parallel partition processing

Today I would like to announce the availability of a little library I’ve written in C# called PartitionCache.

What is it?

PartitionCache is a simple persistent producer partition cache. What does that mean? Well it means it remembers what partition a producer belongs to and allows you to silo your workload into multiple partitions to avoid any multi-threaded headaches or concurrency issues. PartitionCache does not store any events or items that require processing, rather it acts as a persistent store of the partition number each item belongs to.


PartitionCache fills the void of persistent Dictionary where int is the partition and string is the name of the producer. This allows producers or unique keys to have a persistent unique integer assigned to them for their lifetime. PartitionCache gets out of your way and allows you to model your code to eventually migrate to Kafka, Azure Event Hubs or Amazon Kinesis.

I am using PartitionCache to provide an extremely lightweight Kafka like producer partitioner. When processing items you want to ensure that you aren't processing DeviceId1234 items over the top of each other. Perhaps an easy solution would be to use an event streaming software and aggregate the events. In most cases though you just need a quick fix and PartitionCache is just that. You'll be up and running within minutes and you'll be able to increase your processing throughput whilst ensuring items are processed on time.



Download it from GitHub :

Example Client Usage

var topic = "mybackgroundprocessor";

var partitionClient = new PartitionCacheClient(); // Overload for custom URI

// Determine if the topic exists. Topics are created with a default of 16 partitions // You will be surprised how well just 16 partitions will improve processing throughput var topicExists = await partitionClient.TopicExists(topic); if (!topicExists) { await partitionClient.AddTopic(topic); } // Add a producer to topic with key of the person number // The producer name must be something that you aggregate in your processing code // or the producer name can be a Guid or Integer ie. a primary key from a database. var partition = await partitionClient.AddProducer(topic, person.PersonNumber);

Introducing NRepeat. A C# Ultra VNC Repeater

I am pleased to share with you NRepeat. Its something I whipped up over 2 hours to proxy Ultra VNC traffic for NAT routed clients. As long as both ends can see the NRepeat server you can VNC to each other via proxy.

There are plans to make a server GUI, installation program, remote support program, REST API for the server, backend etc.

At the moment there are two samples included in the source code. One is a TCP Proxy, which allows you to proxy a port to another remote endpoint. The other sample is a small footprint multithreaded Ultra VNC Repeater.

You could tailor this functionality for any application essentially.

NRepeat requires .Net 4.5 as it uses the async await features of the framework. You can easily recode NRepeat to use .Net 4.0 TPL if you so desire.

You can download it here: it is released under GNU GPL3.



Removing the VS2013 Update 2 requirement from the Microsoft Native Compilation installer

If you want to tinker with the Microsoft Native Compilation release but you don’t want to update to the RC of VS 2013 Update 2 you can follow these steps:

  1. Obtain Orca (Google it or
  2. Open up ‘netfx_NativeCompilation.msi’ in Orca
  3. Click on ‘Launch Condition’ table
  4. Remove the 2 entries for Visual Studio checks
  5. Click Save


Concurrent Web API requests and session

When using the session in a Web API request you still experience the session state lock. In a typical scenario you shouldn’t be modifying any session variables using Web API requests and if you are its not a big issue to explicitly state that you require to edit the session. Be aware the TempData MVC bag uses the session. If you disable the session state and use TempData you will get an InvalidOperationException.

To set the session state to read only for all requests just add this to your Global.asax

  1.   protected void Application_BeginRequest()
  2.         {
  3.             Context.SetSessionStateBehavior(System.Web.SessionState.SessionStateBehavior.ReadOnly);
  4.         }

On every controller that needs to edit or persist changes to the session decorate the Controller with the following attribute:

  1. [SessionState(System.Web.SessionState.SessionStateBehavior.Required)]

Or you can explicitly ask for read only state:

  1. [SessionState(System.Web.SessionState.SessionStateBehavior.ReadOnly)]


Now all of your Web API requests can run in parallel because they are not requesting a lock on the session state. Smile

Async Await the dirty way–Extension methods

Following up from my previous post you could extend the Lambda Actions and Funcs with extension methods.

  1. public static class Extensions {
  2.     public static Task<bool> AsAsync(this Action method) {
  3.         var task = new Task<bool>(
  4.             () => {
  5.                 method(); return true; }    
  6.             );
  7.         task.Start();
  8.         return task;
  9.     }
  10. }



  1. Action mm = ()=> Console.WriteLine("Hi there, call me sometime?");
  2. await mm.AsAsync();

Async Await the dirty way.

Have a method that you want to run asynchronously but don’t have the time to recode it? Perhaps you just want to iterate a list quickly and return your MVC result using async.

I have made the following snippet to run a Func<T> and Action using the async pattern so you can just await lambda actions and methods. Of course this is purely for quick and dirty situations.

  1. public Task<bool> CreateAsyncTask(Action method) {
  2.     var task = new Task<bool>(
  3.         () => {
  4.             method(); return true; }    
  5.         );
  6.     task.Start();
  7.     return task;
  8. }
  10. public Task<TSource> CreateAsyncTask<TSource>(Func<TSource> method) {
  11.     var task = new Task<TSource>(method);
  12.     task.Start();
  13.     return task;
  14. }

One usage could be this

  1. var y = await CreateAsyncTask(()=> Console.WriteLine("Hello I am going to the console"));

Or you could call a method returning bool

  1. var z = await CreateAsyncTask(() => MyTask(5));

The return await will always be true if you are passing an Action but if you are passing a Func<T> you will be awaiting the resulting T in our case a bool.

Consideration should be made for exceptions but this is just a primitive and dirty solution.

Perhaps an Action lambda variable?

  1. Action mm = ()=> Console.WriteLine("Hi there, call me sometime?");
  2.     var x = await CreateAsyncTask(mm);

Upgrade Entity Framework 4.3.1 to 6.0 EDMX (ObjectContext)

Upgrading to 6.0 from 4.x breaks your build. A lot of the namespaces have changed for the original EF classes and of course some things have been abstracted away.

After a few hours of tinkering I came up with the following solution for painless 4.x upgrade to 6.x Entity Framework. This is useful for people who are still on the older EDMX legacy generation strategy.

What you need to do:

  • Backup everything.
  • Update all projects using Nuget to EF 6 release (you cannot build after doing this)
  • For C# download
  • For download
  • After installing 1 of these extensions you want to backup your EDMX and designer files (or use source control).
  • Restart Visual Studio.
  • After restarting Visual Studio you want to go into your project and:
    1. Get into the diagram design surface
    2. Right click and select "add code generation item"
    3. Select the "EF 6.X Entity Object Generator" and call it something appropriate like ""
    4. Let the generation occur
    5. Go back to the design surface and ensure that the Code Generation Strategy is now T4
    6. Delete the designer file created by the older legacy EF 5.X

This gives you your ObjectContext derived context as well as everything matching the newer EF 6.0 namespaces.

Entity Framework async Await on an Stored Procedure or Function Import

Now this is by no means the best solution. I added an extension method so that I could call await on my stored procedures. In the newer releases of EF6.1+ we should see this officially implemented. Until then a dummy extension method does the job.

  1. static async Task<List<T>> ToListAsync<T>(this ObjectResult<T> source)
  2.         {
  3.             var list = new List<T>();
  4.             await Task.Run(() => list.AddRange(source.ToList()));
  5.             return list;
  6.         }


If you reflect version 6 of EF you will see that ObjectResult<T> actually implements IDbAsyncEnumerable<T>, IDbAsyncEnumerable. And the method for ToListAsync<T>(this IDbAsyncEnumerable<T> source) should be able to wire it up the same as a LINQ query.