Using SWIG to create a .Net wrapper around a C library

A consultant has written a C library to neatly pack images on a space. They have integrated it in their Ruby On Rails / Node.js online app through the power of extending Ruby. I’m quite jealous, because it’s not quite as easy to do in .Net. Especially when, like me, you know absolutely nothing about C/C++.

Two options are available to me:

  • Create a C++/CLI project, that will generate a .Net DLL
  • Use SWIG to generate a .Net wrapper around the C code.

Now, since I can’t change much the C code (it must be built on Mac through Ruby), I thought that I couldn’t easily switch it to CLI (I was wrong ; see the next post).

So I tried using SWIG. And it’s a pain to use. Let me walk you through creating a wrapper around a non-trivial C library.

Generating basic wrapper methods is pretty easy: it’s a matter of creating a .i file with a few lines of code:

Run SWIG on this file to generate a few C# files and a C wrapper class:

Then, you create an empty C/C++ library project with Visual Studio, include all the necessary files, and generate it. It builds a DLL, and you can include both the C# files and the DLL in your .Net project to use. Very nice, very simple.

A problem arises when you have a C method taking an array as argument: SWIG generates a C# signature taking a single instance as argument:

You can tell SWIG to create an “intermediate” array type by adding this to your .i file:

This creates an “image_t_array” class, with methods for setting and getting items, old-school-style. There is also a method to cast it as “image_t”, with pointers handling to go back and forth in the array.

Now, creating an array class, and cast as instance to pass it as parameter is not very intuitive, and I’m trying to create a usable wrapper. I want to pass an actual, managed array to my method. Is that so hard?

The SWIG documentation has a whole part about passing managed arrays to non-managed code. The problem is that (once again), the documentation is incomplete. It shows an example for a managed arrays of ints… but not an array of structs. Apparently it doesn’t work the same way, because SWIG complains about a missing typemap when using this:

Actually, a totally undocumented feature needs to be called just above this line:

Thanks to a nice fellow on StackOverflow for reading the SWIG sources… Googling for “CSHARP_ARRAYS” yeilds exactly zero result in the documentation, but only a few relevant results on forums and mailing lists. Why is it not documented, while it’s obviously necessary? If it’s not the right way, which is? Why not documenting the warning message that appears when you don’t use it?

So anyway, now my C# method takes a managed array as argument, which makes this library so much more easy to use. It throws a memory access exception when running the “pack” method, though (System.AccessViolationException).

Thankfully I have both the source files, and the pdb file for the unmanaged dll. I just have to copy the pdb to the target directory (using my test program post-build events), and once I activate unmanaged code debugging (managed project properties / debug  /activate unmanaged code debugging), I can step through the library, using the actual C code! Awesome.

At the actual point that an error occurs, I can see that the library tries to access an image in the array that is not defined. Weird. By adding a few breakpoints in the C library (I can’t imagine how I would have debugged that without the source code…), I quickly notice that my “*images” (in the pack) have very weird values, and that they do not correspond to the sent values at all.

Since passing values as arrays might do weird things with pointers, I decide to switch back to using the “image_t_array” class… and sure enough, now the values are correctly passed. I still have memory access problems, though, but that seems to be on the C side… but it runs nicely when a C++ program calls it, so I still have some digging to do.

At this point, debugging the wrapper becomes very complicated with all the layers involved, so I decided to try the C++/CLI way which, as you may see in my next post, is not that hard after all.

Use git with your SVN repository

You are just a few simple commands away to start using all the local power of git (local commit and rewind, stash…) over an existing SVN repository.

I say local power, because obviously, SVN being not as powerful as git, you’re still limited in what you can do without breaking everything when you try to commit.

A simple and effective guide can be found here : http://svnrating.com/svn-to-git-migration/

Note that on large SVN repository, the first checkout/clone can take hours, or even days. On our 2400-commits-large, several-Gb repository, the first clone has taken a few days, but it was making my (bi-Xeon + 16Gb RAM…) computer hang a few seconds or minutes every 15 minutes or so. Maybe you’ll have better luck.

But once it’s done… it’s really awesome.

Linq2SQL – Row Not Found Or Changed

Working on a .Net 3.5 client app, I am stuck using the lesser evil of Linq2SQL (compared to the awefulness that is EntityFramework 1).

Updating data on a column results in a ChangeConflictException : “Row not found or changed”. The usual culprit of the DBML differing even slightly from the actual DB model was not to blame, and the concurrency was not either, since the processing took a few milliseconds at worst, on a local DB.

It actually was a trigger updating the “updated_at” column. In the DBML editor, setting the column to “check update : never”  and “automatic synchronization : always” solved the problem.

Listen to your users

Why would I want to?

Why are we writing software ? Because we need, or we like, to solve problems.

I you write software for people to use (as opposed to embedded software, for instance), these problems will only be solved if these people actually succeed in using this software. This is, both surprisingly and annoyingly, incredibly difficult.

Your users are not dumb. They are not as well-trained as you are in using computers. They often will be far superior to you in their field of choice. Don’t underestimate them.

And even if you do underestimate them, or even if your users were actually not-that-smart, so what? Aren’t you writing software for them anyway? You are not your user.

How do I do that?

I urge you to do user testing. It will blow your mind, and might make you rethink your whole software. It doesn’t have to be huge, you just need to put people in front of your software, and watch them use it.

You can begin by asking co-workers that are not as familiar as you are with the product : a secretary, an accountant, a salesperson… If you’re in a building with other companies, you might ask people there. Better yet, if you can, ask real users (or prospective users) to come to your office.

Get in a room with them, get them on a laptop with a mic (most do, these days), install a capture software like BB Flashback Express, run it, then run your software or open your website.

Then, just ask the users to do common tasks. Insert this, edit that. Don’t help them. You might find plenty of help everywhere on how to conduct user testing.

How will it help me?

Listen to their remarks. Probe them for more informations. Don’t necessarily do what they suggest, but think about what problem they encounter, what problem they want to solve with their suggestions, and how you would solve it optimally.

Some examples of recent user comments that have had a tremendous impact on our recent designs :

I think I understand how to resize this block, but why does it display a “hand” cursor instead of an arrow ?

This allowed us to realize that our block resizing was even more awful than we thought, and we redesigned them completely. The blocks had arrows on the corners that allowed the users to resize, and on mouse-over the cursor changed to the “link click” cursor. The common pattern is that the cursor must change to a “double arrow” to resize. We were so used to this anti-pattern that we didn’t even notice it anymore. Now users finally understand how to move and resize blocks !

I think the previous version was more simple to use. I liked the toolbar.

This comment made us realize that our “clever” new design was in fact very confusing. The new design uses contextual panels and buttons in various places to do various actions, it’s kind of pretty, and has a lot of attention to details. The old one was using a single toolbar for all actions, was awfully ugly, with widely inconsistent icons that changed places depending on what you clicked. But users always knew that the actions were there. Now they’re lost, because the action to do X is on the left, Y is on the bottom, and Z is displayed in a contextual panel. It’s only this comment that made me realize why I was uncomfortable when watching users search for some actions: they are actually lost. We might have to rethink our whole design to switch to toolbar-based.

That’s it?

Of course not. It’s very easy to do enhancements with just that, but it won’t allow you to build the best software out there. But there is some help available everywhere.

I personnally started with Steve Krug’s “Don’t make me think” and “Rocket Surgery Made Easy” : they are very concise and to the point. I’d consider them mandatory readings.

There are many great usability blogs out there: try out (in no particular order) Smashing Magazine, UX Magazine, UXMatters, Boxes And Arrows, The UX Booth, UX Movement, Usability Counts, Wireframes Magazine… the list goes on.

I am sure this small start will help you a great deal.

Abstract away constructor logic

I need to create an administration interface in MVC, over an old and seriously crappy DB. The fields and tables have no consistency whatsoever, but that’s easily solved through renaming things with Entity Framework. But now, I need to do 5 administration interfaces for 5 trios of tables : “master”, “images” and “categories”. They are pretty much the same, except the table and field names. To do that, let’s play with abstraction.

I’m using crude ViewModels to display my search pages. They get the elements to display in the search dropdown lists, and paginate the results. All of the logic is in the constructor (the viewmodel is only used to display a search screen). When the constructor needs to get specific elements, it calls abstract methods. They are the ones that are actually different in the various pages.

The abstract base looks like this :

And now the implementation is very simple ; here for the themes :

Theme_Combination is the joining of two entities, Master and Images. All your element_combination must inherit from IElementCombination. If they do not implement anything specific themselves, you can even use a single ElementCombination class.

Bug in WPF 4 text rendering

Microsoft has completely changed the text rendering engine from WPF 3.5 to 4.0. In the meantime, it also has introduced a bug, which you can read about here : http://stackoverflow.com/questions/23246254/ and here : https://connect.microsoft.com/VisualStudio/feedback/details/860053/wpf-4-font-rendering-bug-with-some-fonts-generated-xps-is-invalid. I have built a test project that demonstrates the problem here : https://github.com/tbroust-trepia/wpf-4-font-rendering

What looks like a small problem (the apostrophe character ” ‘ ” actually reduces the space between two characters with some fonts, resulting in overlapping characters) is problematic when you generate an XPS file from the XAML, because the resulting XPS is “corrupted”, with negative letter-spacing, which seems illegal in XPS.

I don’t really know which is to blame more : the font rendering bug, or the XPS converter (integrated in .Net !) which renders the text faithfully, but shouldn’t. Anyway, while waiting for MS to fix anything, I have found the way to fix it myself. Warning: dirty hack ahead.

Update: Microsoft has closed the ticket, and, obviously, they have decided to not resolve the issue, since it impacts such a small number of people (maybe just us ?). I always thinks it’s sad when a company this big does not close such a small and quickly-fixed bug.

WPF and MVVM discovery

In our setup, we have operations that cannot realistically be run by the installer itself, so we have a separate application that updates various components at the end of the setup. I am rewriting this app so it’s cleaner (which isn’t very difficult considering the current state), and taking this opportunity to familiarize myself with WPF and the MVVM pattern.

I am using MVVM Light, and it’s a bit of a pain. There is exactly zero official documentation. It’s a real shame, considering that it’s one of the most used MVVM frameworks. I would love to use Caliburn.Micro because it’s well documented and supported, but using it in conjunction with MahApps.Metro adds yet another layer of complexity, and I am already learning about many things at once.

Creating a responsive UI in WPF using MVVM involves many steps and components. I will need to do my work in a BackgroundWorkerautoscroll a text box, etc, and bind it in my MVVM view. What I can find here and there often lacks a few important points, but it’s generally easy to find help.

Your ViewModel needs to declare public properties for each of the things displayed in the view, that you might want to change in the ViewModel. For instance, with a progress bar, if you don’t know how many steps you will have in advance, but you don’t want to use percentages, you will need to bind the Maximum property in the VM. Opening message and confirmation boxes should be done in the view, otherwise your VM unit tests will open dialog boxes that you can’t close. To do that, you need to create a messaging system.

All in all, I still don’t like WPF/MVVM very much. It’s very verbose, and doing anything takes a huge number of steps, namespaces importing, classes, workers, commands, and messaging back and forth. I have yet to come to a point where I find that all this overhead is useful. WPF seems really powerful, but it’s also really complicated and verbose to do anything at all. The overhead of adding a screen is huge: you have to create like a thousand properties, relay commands, events and events handlers, etc. I get that it’s the right way, but it’s a really long and windy way.

The main problem I have seems to be that people come up with incredibly complicated solutions to simple problems, and then say stuff like “it is easy to do X” and “simply do Y”… after posting a hundred-lines-long post with half a dozen classes… to display a message box.

As an aside, I hate Windows XP. In addition to being incredibly buggy (most of our customer’s problems come from XP users), it prevents us from upgrading to .NET 4.5, and using shiny new toys. It’s like we’re stuck in the dark ages. We have almost 10% of our customers still using XP. It’s huge! And about 40% of them seem to never update their software version, even though they are asked to do it at each software startup. That’s a bit depressing. We can’t use the C# async/await keywords, and more and more NuGet packages require this version. I can’t even find how to display a MahApps.Metro dialog box without the async keyword.

Exploring various .NET build solutions: Albacore vs FAKE vs PSake

As I said in a previous post, I am changing our build process for our application, so I’m testing several build solutions.

FAKE is the newest and shiniest tool for .Net users. F# looks like a cross between Ruby (for the general syntax), and PowerShell (for the piping-fever-dream). Unfortunately, it seems to be missing a core piece of any proper build system: running an arbitrary exe file. I need to run the setup builder (InnoScript), but all I can find is creating a custom task, and it’s way overkill, and not very evolutive. Documentation is incredibly hard to find because of its generic name. Pro tip: do not name your software after a random generic word.

Albacore is great, running on Ruby, using Rake tasks. Installing Ruby on Windows is easy, as long as you don’t try to use some gems that won’t compile on your system. Unfortunately, Jenkins installed on Windows won’t run the ruby installed with Ruby Installer, for various reasons: PATH problems, rights problems, etc. Too bad, because my build script was working great.

PSake might be the easiest to use on a Windows build system. Documentation for PSake itself is very light, but Powershell is very well documented, very powerful, and we already use it in various places in our systems, so we know it pretty well.
The only thing I don’t like with Powershell is that, to keep things simple on one side, you have to do complicated things on the other. I don’t want to install a thousand plugins in Jenkins, so I need to build a batch “boostraper” that runs the PS scripts.

The good thing is that PSAke and Rake are very similar, so I was able to write the equivalent script in a few hours, just by putting my Rakefile and build.ps1 files side-by-side in Sublime Text.

Update

After a year working with PSake for all of my build scripts, I can confirm that it’s awesome. Powershell is a great automation language, it’s very mature, and it has thousands of libraries and extensions.

Rails is like magic…

… but more like black magic. The one where you have to sacrifice kittens to get anything done.

I want to do a simple thing using devise: authenticated users go to a page on the root of the site, and unauthenticated users go to another.
I’ve been banging my head for days on this. Between the various ways to do it, the different ruby, rails and devise versions that use different methods, it’s a mess. Nobody seems to agree on the best way to do it.

Well, I’ve found the way that I believe is the most simple.

Things to remember :

  • “devise_scope” is necessary. I do not understand why, nor what it does.
  • the “as: xxx” parts are necessary, too. It looks like Rails is fine with creating two routes pointing to the same URL, but wants different names for them. I would have tought that it would create these routes only at run-time.
  • do not modify several things at once and hope to understand what went wrong when everything crashes down, as long as you don’t understand the language and framework.