Graphics vs Resolution Independance

I love my retina iPad (and retina iPod Touch, but no retina MacBook yet) and I hate seeing applications that does not support them, it’s wasting pixels. However I’m not exactly an artist – as seen in yesterday’s screenshots.

In that particular case I think the “tv” text makes a great icon – even if it may be a bit too localized (is AppleTV translated to something else anywhere ?). Anyway it’s clear to me that using a bit of code, like below, to generate images is a good way to support any number of screen resolution / density for mobile applications.

	static UIImage GetIcon ()
	{
		float size = UIDevice.CurrentDevice.UserInterfaceIdiom == UIUserInterfaceIdiom.Pad ? 
			55f : 43f;
		UIGraphics.BeginImageContextWithOptions (new SizeF (size, size), false, 0.0f);
		using (var c = UIGraphics.GetCurrentContext ()) {
			c.SetFillColor (1.0f, 1.0f, 1.0f, 1.0f);
			c.SetStrokeColor (1.0f, 1.0f, 1.0f, 1.0f);
			UIFont font = UIFont.BoldSystemFontOfSize (size - 8);
			using (var s = new NSString ("tv"))
				s.DrawString (new PointF (7.5f, 0.0f), font);
		}
		UIImage img = UIGraphics.GetImageFromCurrentImageContext ();
		UIGraphics.EndImageContext ();
		return img;
	}

This technique was refined by the folks at PixelCut, creator of PaintCodeApp, which let you create CoreGraphics-API compatible source code (either in Objective C or C#) from images you draw (instead of type).

This application is fantastic unless you realize you’re lacking a minimum of artistic talent Thanksfully other people have this talent, like the folks doing FontAwesome which offers a lot of nice, useful icons in several formats.

One such format is SVG, specifically SVG paths, which is something we had to implement in Moonlight (that seems a lifetime ago). Translating this C++ code into a C# library was pretty easy. In fact it was a lot easier than doing the original code – parts of the spec are not trivial, e.g. you can find quite a few SVG (path) library or converters that simply ignore arcs (and even other constructs).

Next, using this new library I created a small (less than 100 lines) tool to convert all symbols into a (rather big, around 15k lines) C# source file containing all SVG paths, each in it’s own method. That generated file was then used in a MonoTouch sample app (again less than 100 lines) to show them in a table view (using MonoTouch.Dialog).

Now this is not really new, Touch.Unit has been using this for a while. New or not, consider this as my xmas gift to other art-challenged developers in needs of nice, scalable icons for their mobile apps

Also if you’re not using C# or CoreGraphics then you might want to contribute other backends. The library (and command-line tool) are made to be extensible – but I do not have immediate needs (or plans) for other language / toolkit at the moment.

AirPlay vs iOS API

This post should not be a big surprise. As you might have guessed I had other ideas with my AppleTV.

I think there’s a large, untapped potential in the AppleTV to be used with/from other devices. Collaboration, visualization and of course gaming (think of the Wii U) comes to mind. Sadly several pieces are missing for this to become reality Let’s try to add one of them…

If you played the iOS API for AirPlay you already know it does not really support pictures, i.e. what the (Apple supplied) Photo.app does is not available to developers. In particular:

• The device selector (available thru MPVolumeView) shows every AirPlay devices, including AirExpress for speakers. Photo.app somehow filters them since they can’t show pictures;
• It does not allow you to provide what’s to be shown on the AppleTV, that must be done with the MPVolumeView which is not picture friendly;
• The closest you can get is screen mirroring (only on recent iOS devices) and that’s not quite the same since your (iOS device) screen is not available to other uses.

Note: I know private API exists to do this but, like the jailbreaking, I do not want my app to depend on them.

Like I showed before using .NET to show pictures to an AppleTV is quite simple. What’s needed is a UI for this…

And here’s come Poupou.AirPlay assembly that provides browsing AirPlay devices and sending pictures to them. It works on any iPad and iPhone/iPod Touch (at least everything running iOS5, I did not test earlier versions – feedback appreciated).

To select a AirPlay device the code uses UIActivity, available in iOS 6 (and later).

iOS 6.0 screenshot

But it’s easy to support earlier iOS releases with a bit of extra code. The sample application shows you how to do that.

iOS 5.1 screenshot

AirPlay vs Large Digital Frame

We (as a family) have been looking for a large digital frame. My wife takes a lot of pictures (mostly kids and vacations) and showing them was always… less than optimal.

• We have a few, small 7-8 inches, digital frames in the house – but we do not update them very often. Copying files around SD cards is just not fun enough;
• The frames have a lot of options but we can never get the pictures to be shown long enough for our taste. That would be closer to 10 minutes and definitively not a 30 seconds slideshow;
• The most common alternatives are to show pictures on the TV (we still have to get the pictures there) or sit in front of the computer (no file copying but not confortable for many people). We don’t do that unless we have friends or family visiting us;

So we wanted something larger with the right options – the kind of set and forget.

It turns out there are not many large digital frames out there. Most of them are using computer monitors with additonal hardware, sometime an (jailbroken) AppleTV. The biggest addition, to the common hardware, is the price markup on them.

Now in a totally, until recently, unrelated subject I’ve been a fan of the AppleTV for a while. However the sad truth is I don’t use my AppleTV. Why ? I can’t (yet?) create apps for it and bandwidth caps, which are almost universal in Canada, makes Netflix less attractive (and I have a dozen Netflix compatible devices anyway).

Does a large (e.g. 23 inch) monitor and an AppleTV makes everything fine ? Not quite.

• We do not want to move pictures (to the cloud or a device). If it takes extra steps then it won’t be done regularly enough to make it useful. The files are already in the computer/network drives;
• We want the picture to be shown for more than a few seconds. The only right predetermined time is the one we decide.

The AppleTV itself does not do that and, for various reasons, I did not want to jailbreak it. Thanksfully the AirPlay protocol was reverse engineered so it is possible to send pictures from a computer to an AppleTV. The device was not bought with this in mind, nor did I look for the AirPlay protocol only for this, but it turned out really easy (less than 100 C# lines) to make it work just like we wanted, which is:

• Scan a directory (and subdirectories) for pictures;
• Shows pictures randomly – but never twice before starting again (by re-scaning the directories);
• Send pictures to the AppleTV and wait for X seconds before showing the next one, i.e. the program, not the device, control the delay between pictures.

Curious ? it’s in github

Touch.Unit vs NUnitLite 0.7

Last week Charlie Poole released NUnitLite 0.7. This new release greatly reduce the feature gap between the older 0.6 release and the upcoming NUnit 3.

I know people will rejoice having Assert.AreEqual(x,y) back as it is simpler than the Assert.That (x, Is.EqualTo (y)) syntax. Personally I’m more happy about Assert.Throw as I really like this one over the [ExpectedException] attribute. There’s a lot of new features / attributes, many I never used (since the Mono-shipped version of NUnit did not have them), that should prove useful.

At this stage the only missing piece, IMHO (but shared with others too), is the lack of async testing – which I grown fond of during Moonlight‘s development.

Anyway this post is to announce that Touch.Unit was updated to use this new 0.7 version of NUnitLite. Existing (old) features should be working fine (MonoTouch bots seems quite happy with it) but I have not yet used (tested ?) most of the new features. If you find anything wrong please fill a bug report !

Availability: The next version of MonoTouch 5.3.x will provide the updated Touch.Unit runner but if you can’t wait (or update) then pull the update from github and “test” it away

Linker vs Bindings and IsDirectBinding

If you looked at my previous entries related to the linker and bindings: NewRefcount, UI thread checks, Runtime.Arch or at MonoTouch‘s generated bindings source code then you likely noticed another common pattern, e.g.

	[Export ("sizeToFit")]
	[CompilerGenerated]
	public virtual void SizeToFit ()
	{
		global::MonoTouch.UIKit.UIApplication.EnsureUIThread ();
		if (IsDirectBinding) {
			Messaging.void_objc_msgSend (this.Handle, selSizeToFit);
		} else {
			Messaging.void_objc_msgSendSuper (this.SuperHandle, selSizeToFit);
		}
	}

The use of IsDirectBinding (line #6) is a bit less common in third party bindings, unless the -e option was used with btouch to allow types to safely inherit from your bindings. The fact that such an option exists is a sign that we’d like to get rid of this condition when it’s not required.

If we knew that a type is never subclassed then we could remove the IsDirectBinding (line #6) as its value would be constant, true, and that would also allow the removal of the false branch (line #9).

Along with some previous optimizations that can remove [CompilerGenerated] (line #2) and EnsureUIThread (line #5), we would end up with a smaller, branchless method that simply calls the right selector, like this:

	[Export ("sizeToFit")]
	public virtual void SizeToFit ()
	{
		Messaging.void_objc_msgSend (this.Handle, selSizeToFit);
	}

Now that’s something the linker can find out since it must analyze the application’s code. Furthermore it also knows, since runtime code generation is impossible on iOS, that the application cannot be extended, unless it’s re-compiled (and re-linked). That’s turning a disadvantage into an advantage! and, starting with MonoTouch 5.3.3, the linker gives you this (small) consolation

This optimization saves 22 kb on a release build of TweetStation but, like most other linker-bindings changes, the real goal was not to save space (it’s still nice) but to reduce the number of conditions and branching in the code to allow faster execution.

Managed Crypto vs CommonCrypto : Deciphering results

After the hash algorithms the next part of MonoTouch‘s switch to CommonCrypto covers the symmetric ciphers, commonly referred to CommonCryptor. This includes:

• DESCryptoServiceProvider, TripleDESCryptoServiceProvider, RC2CryptoServiceProvider and parts of RijndaelManaged inside mscorlib.dll;
• ARC4Managed inside Mono.Security.dll; and
• AESManaged inside System.Core.dll

Those are all the symmetric algorithms that Mono provides (using managed code). Now they will all, for MonoTouch, be native except for the non-AES parts of Rijndael, i.e. when its block size is not the default 128 bits.

This time hardware acceleration is easier to detect as we can compare AES Electronic Codebook (ECB) mode with AES Cipher Block Chaining (CBC) mode – the later documented to be accelerated. In software the most basic mode is ECB and the other modes are built on top of it. So ECB is generally a bit faster than CBC. If CBC turns out to be faster then it was either further optimized (small margin) or hardware accelerated (large margin).

The above graphics, made from the iPad 3 results, shows the performance (MB/s) on the vertical scale versus the buffer size (bytes) on the horizontal. Unlike the digest results it’s hard to suspect anything other than hardware acceleration for such a drastic difference.

The next graphics compares the performance of the native/hardware implementations (of CommonCrypto) versus the managed implementations (of Mono). Values are the average, in MB/s, of all my devices. You’ll see that using older algorithms, like DES or TripleDES, is not the way to better performance, managed or native.

Wait! where’s the 35.4x performance increase ?

Uho, the AES results looks more like 10x than 35x, right ? That’s (a bit) because I used the average values of my devices and (mostly) because I did not use the magic tricks. IMO that gives a more honest picture – at least as much as benchmark can be

But, as promised, here are the exact incantations and sacrifices required to get up to 35x increase. You’ll need three things:

1. An iPad 1st generation. It simply has the best performance – well over the iPad3 at 18.8x and iPod4 at 25.2x (see graphics below). It’s large and surprising variation. Yet all results are better than the non-magical 10x average;

2. Your device needs to run on AC power. That will give more juice to the crypto processor – roughly doubling its performance. Power management, under battery, does not allow this level of performance. That’s part of the sacrifice required to get the maximum performance;

3. Use the optimal buffer size (again) but this time we’re talking about huge buffers. In case you did not notice them, on the ECB/CBC graphic above, you will need 512KB to get the maximum throughput (86.9 MB/s on iPad1). If you drop the buffers to 256KB you lose more than 3 MB/s. Drop them to 128KB and you’re down to 68.1 MB/s. Drop to 16 kb (the optimal size for battery operation) and you’ll only get 44.1 MB/s – a long way from the 86.9MB but still better than 29.4MB/s (same buffer on battery). Keep in mind of the I/O required to keep the buffers full – it might be easier (and finally faster) to use smaller ones…

Running iOS application on AC power is uncommon for most people. However it’s not uncommon when developing and testing applications, including benchmarking them. Take care when testing yours! It took my a while and a bit of juggling between computers and devices to figure out why some numbers were so different than others.

Here is the new, battery operated, and the original (from the first blog post), AC powered, performance graphics.

Managed Crypto vs CommonCrypto : Digesting results

As part of MonoTouch‘s switch to CommonCrypto most of the digest (hash) algorithm implementations were changed to use CommonDigest. This includes:

• MD5CryptoServiceProvider, SHA1[CryptoServiceProvider|Managed], SHA256Managed, SHA384Managed and SHA512Managed inside mscorlib.dll; and
• MD2Managed, MD4Managed and SHA224Managed inside Mono.Security.dll

If you know well the System.Security.Cryptography namespace then you’ll note this is every HashAlgorithm except RIPEMD160Managed and all the extra ones Mono provides to support (mostly older) X.509 certificates.

This also has some indirect effects on some other types, e.g. all HMAC and MAC (e.g. MACTripleDES) implementations are using the default hash algorithm. This means that all of them, except for HMACRIPEMD160, are now using CommonCrypto and will be faster too.

Note: Mono never followed the [Managed|CryptoServiceProvider] suffixes: by default everything was always managed and will now be native (for MonoTouch). So don’t be confused by the names. Source/binary compatibility requires Mono type names to match Microsoft, not how they are implemented.

Performance

The best performance comes from SHA1, 107 MB/second (mean values for my iPod4, iPad1 and iPad3 devices). This is also the best enhancement ratio, 6.7x faster than managed – even if SHA1 is the most optimized (i.e. unrolled) managed implementation Mono has.

The secret ? hardware acceleration. Apple’s source code, shows that hardware acceleration is being used (at least on some ARM devices) if the block being processed are large enough (to offset the cost of initializing the hardware).

It’s always a bit hard to be 100% certain we’re using the hardware as there’s no way to ask for (or against) it or even query it’s use / availability. In this case the numbers do not make it very clear, the curves are too similar between all native implementations to jump at a conclusion. My own gut feelings is that we’re seeing the effects of hardware acceleration but it lost its shocking effect because it’s compared to Mono’s best, performance-wise, implementation.

Getting the most of the update

The above graphic shows considerable performance gains: from 3.1x up to 6.7x. Now the question is: can we get such gains inside our applications ?

Without surprise it comes back to selecting an optimal buffer size (and that advice is true for older MonoTouch, Mono itself and other Mono-based products). This is not always as easy as it sounds (e.g. SSL/TLS) but if you can control your buffer size (and measure) then you can get the best performance for your application.

The next graphics shows the performance (managed versus native) of SHA1. There’s no gain to use CommonCrypto before reaching 16 bytes but surprisingly there’s no loss either. That’s unlike the /dev/crypto results and a very good news since it means this update should not degrade performance for existing applications.

Another similarity: in both cases the optimal buffer size is (close to) 4096 bytes – and that’s also true for other algorithms as well. That should not be a huge surprise for managed code, since it’s the default value Mono has used for years when computing the hash on a stream.

This is a (second) good news that the managed and native (and hardware) optimal buffer size are close enough so a single value can be shared. It simply makes coding easier for everyone

So that’s 6.7x increase is for real ? Not quite. Did not you hear me repeating benchmark are lies ?

Benchmarking cryptographic implementations can be (and generally is) done with minimal I/O. Memory (RAM) is much faster than disk (or any type of storage) and faster than networking. Beside the I/O times this also remove/reduce the need for memory allocations (and the related GC time).

OTOH your application, if it’s not a benchmark apps, will need to do a lot more I/O (loading, saving or transmitting the data) that will result in memory allocations… all of which will reduce the performance gain. Also the faster the cryptographic implementation becomes the less forgiving it will be for I/O times.

So your mileage will vary. Someone already reported gains near 3-4x, which is quite impressive inside a real application. IMO the best news is the lack of scenarios where things gets worse – it should be a win for everyone