Monday, November 4, 2013

Using Xamarin Sharpie to build a Flurry assembly

The main project I work on is a series of Xamarin based apps for iOS devices.
To help track usage stats we use Flurry Analytics which gives us a handy dashboard showing us session usage, device & OS breakdown, error logs, etc.

With the iOS 7 release, Flurry required a update:

Flurry doesn't have a .Net assembly out-of-the-box, so we needed a way of converting the native library into .Net.

When we had added Flurry to our project ages ago, we started with the Flurry monotouch-bindings to get things started. However this time Flurry had changed something in their library and the monotouch-binding no longer worked correctly. Trying to run the app with the built assembly resulted in build errors including:
Undefined symbols for architecture armv7:
  "_SecItemAdd", referenced from:
      -[FlurryKeychainWrapper setData:forKey:] in libFlurry.a(libFlurry.a-armv7-master.o)
  "_SecItemCopyMatching", referenced from:
      -[FlurryKeychainWrapper dataForKey:] in libFlurry.a(libFlurry.a-armv7-master.o)
  "_SecItemDelete", referenced from:
      -[FlurryKeychainWrapper removeObjectForKey:] in libFlurry.a(libFlurry.a-armv7-master.o)

Missing "sec" stuff. Doh!
I'll walk you through what I had to do to sort this out.

What's changed in Flurry?

Before starting, I had a quick look at the release notes to see if anything jumped out:

Hmmm…. point 4…. a new "Security" framework was added. Best keep that in mind for when we do the linking later.

Building a Flurry assembly

Firstly, download the native SDK from Flurry. You will end up with the following files as part of the sdk:

Next, we will need to convert the ".a" native library into a .Net assembly. Head over to Xamarin and download Sharpie. This useful tool will parse the Flurry.h header file and output a C# binding definition.

Follow through the steps in the tool and you'll end up with a binding file which will include something similar to:
using System;
using MonoTouch.Foundation;

namespace FlurryAnalytics {

     public enum FlurryLogLevel : [unmapped: unexposed: Elaborated] {
          None = 0,
     [BaseType (typeof (NSObject))]
     public partial interface Flurry {

          [Static, Export ("appVersion")]
          string AppVersion { set; }

          [Static, Export ("getFlurryAgentVersion")]
          string GetFlurryAgentVersion { get; }

          [Static, Export ("showErrorInLogEnabled")]
          bool ShowErrorInLogEnabled { set; }

          [Static, Export ("debugLogEnabled")]
          bool DebugLogEnabled { set; }
          [Static, Export ("sessionContinueSeconds")]
          int SessionContinueSeconds { set; }

          [Static, Export ("secureTransportEnabled")]
          bool SecureTransportEnabled { set; }

Its done a pretty good job, but has just gotten a bit confused with the enum 'FlurryLogLevel'. I took that enum out of the binding file and added it to a new "extras.cs" file I created:
namespace FlurryAnalytics
     public enum FlurryLogLevel
          None = 0,

I now had the following four files in the folder:

Next, we will need to give some build instructions to Xamarin to tell it how to handle building and linking the native library and the binding definition. To do that, we add an AssemblyInfo.cs with the following content:
using System;
using MonoTouch.ObjCRuntime;

[assembly: LinkWith ("libFlurry_4.2.3.a", LinkTarget.Simulator | LinkTarget.ArmV7, Frameworks = "SystemConfiguration Security", ForceLoad = true)]

The linker is going to be told to grab the libFlurry_4.2.3.a library, link in the simulator and armv7 targets (armv7 is for the devices) and include the SystemConfiguration framework. Notice also that we included the Security framework. That should take care of the build errors.

Lastly, lets take the monotouch-bindings 'makefile' and alter it to better match our needs:

all: FlurryAnalytics.dll

FlurryAnalytics.dll: Makefile AssemblyInfo.cs FlurryAnalytics.cs libFlurry_4.2.3.a
     $(BTOUCH)  FlurryAnalytics.cs AssemblyInfo.cs extras.cs --out=$@ --link-with=libFlurry_4.2.3.a,libFlurry_4.2.3.a

You'll now have the following files:

Open up a terminal, go to the above folder and run the build by typing "make". A FlurryAnalytics.dll should appear in the folder. You can include this in your project and should be able to start using the Flurry magic.

Wednesday, October 30, 2013

Intro to using the Windows Debugger

Windows Debugger can be downloaded as part of the Windows SDK. I would install both the 32 and 64bit versions.

I used it in the past when diagnosing why a website was performing poorly during peak periods. Users would hit the website url, wait for 30+ seconds and then the site would appear quickly. Even the home page which did nothing (i.e. no db hits), was slow to appear.

I got a memory dump from the server and using the analysis approach below could see that all available IIS worker threads were in use and all were busy rendering. The server was starved of threads and user requests were ending up in a queue waiting to be processed.

I tweaked the IIS settings and added more worker threads. The problem went away. Thanks Windows Debugger!

How do you to get a memory dump?

1. Open a command prompt
2. Get the list of worker process ids:
  • %systemroot%\system32\inetsrv\APPCMD list wps
    • Your worker process should be in the list
3. Change to the Debugging Tools directory. On my machine that was at:
  • cd C:\Program Files (x86)\Windows Kits\8.0\Debuggers\x86 4. Create a directory for the memory dump. I went with c:\dumps
5. Run the following, substituting your process id for xxxx:
  • ADPlus -Hang -p xxxx -quiet -o c:\dumps. You'll end up with a .dmp file in the c:\dumps folder.

Opening the memory dump in WinDBG

1. Run WinDBG. You'll see something similar to the following:

2. Goto File -> Open Crash Dump. You should then see something similar to the following:

3. Set the symbol path:
  • .sympath srv*c:\Symbols*
  • .symfix
  • .reload 

4. Load the .Net CLR
  • .Net 3.5: .loadby sos mscorwks
  • .. or for .Net 4.0: .loadby sos clr

Working out what each of the threads are doing

1. Get the list of threads in the memory dump
  • !Threads -special 

2. Switch to a ThreadPoolWorker thread, e.g. thread 11:
  • ~11s 

3. Get the stacktrace for this thread
  • !clrstack 

Using the stacktrace, you can then see what the thread is doing. Switch then to a different thread to see what it is doing. Using this approach I could work out that all threads were busy doing normal activities and that the website just needed more worker threads.

MVC HTTP Headers for CDN caching

On a recent project I added upgraded our systems to use a Content Delivery Network (CDN). The project supplied content to users and we wanted to offload the 'heavy-lifting' of the content supply away from our PROD server onto a more robust delivery network.

The mobile clients would request the content from the CDN. If the CDN hadn't cached the content yet, it would grab it from our PROD server, cache it, and respond back to the clients. For this caching to work, though, some HTTP Header magic was required.

Which headers should you watch out for?

  • Cache control should be public
    • For the CDN to cache the content on behalf of your origin server, set the Cache-control=public
  • Expires Date
    • To stop the CDN regularly requested the same static content from your PROD server, set the Expires-Date setting to be in the far future (e.g. > 1 year)
  •  Vary 
    • Watch out for the 'Vary' header. MVC3 had initially defaulted to adding the "vary=*" http header which stopped the CDN from caching the data. 

A useful RFC to refer to is: RFC 2616 13.6:
A Vary header field-value of * always fails to match and subsequent requests on that resource can only be properly interpreted by the origin server.

.. or if you prefer the MSDN page:

If you set Vary="NONE", this gets outputted as "Vary=*", which isn't very useful. Either get rid of the Vary header or set it to something broad such as "Vary: Accept-Encoding"

How to implement this in MVC?

[OutputCache(Location = OutputCacheLocation.Any, Duration = 31536000, VaryByParam = "")]
public ActionResult GetMyCDNContent(string someSortOfIdenfier)
OutputCacheLocation.Any will be outputted as: Cache-Control: public
Duration = 31536000 will be outputted as:
  • Cache-Control: max-age=31536000
  • Expires: {a date a year from today}
  • VaryByParam="" will not output any Vary= header
  • This is what we needed; the content was going to be the same for all clients.

Here's how it ends up looking in Fiddler:

You might want something different if you have alternate caching needs for different encodings, such as support for gzip and non-gzip caching:
[OutputCache(Location = OutputCacheLocation.Any, Duration = 31536000, VaryByParam = "", VaryByHeader = "Accept-Encoding" )]
public ActionResult GetMyCDNContent(string someSortOfIdenfier)

Here's how it ends up looking in Fiddler: