Saturday, May 10, 2014

Azure Continuous Delivery Build Configuration

Setting up an Azure website with continuous git deliveries from Visual Studio Online was straightforward enough but it wasn't entirely clear to me how you specify which Build Configuration to use (e.g. Debug vs Release). It turns out to resolve this:

1. Open your site up in Azure Management Portal


2. Click on the Visual Studio link (bottom right of the above screenshot). Studio should then open up

3. Within Visual Studio's Team Explorer Tab, click on the Builds button


4. Within the Builds tab, you'll see the build definitions. Interestingly, behind the scenes Azure created a "{site}_CD" build definition


5. Right click on the _CD build definition and select "Edit Build Definition..."

6. On the property page that appears, select the "Process" tab


You can then edit the Configurations and select the Build Configuration you want

Friday, May 9, 2014

Amazon S3 Bucket custom DNS name

Amazon's S3 is an incredibly useful service for storing files and blobs of data. The data is stored in Buckets which are accessibly via an Amazon generated url, similar to:

http(s)://.s3.amazonaws.com/
or, for example, 
http://myFabBucketExample.eoinclayton.net.s3.amazonaws.com

The url does end up being a bit unwieldy though. You may also want to cover the potential future scenario of moving away from S3 and not having to deal with amazonaws.com urls littered throughout your code bases.
To deal with this, if you already have a DNS name (e.g. eoinclayton.net) you can map it to your bucket. To achieve this:

1. Create a bucket in S3 with the same name as you would like your DNS name to appear. E.g. I wanted to end up with a new "assets" subdomain, e.g.  http://assets.eoinclayton.net/{somefile}, so I made a bucket called "assets.eoinclayton.net":


2. If you want the bucket contents to be viewable by the public, modify the S3 bucket so that 'Everyone' has 'Open/Download' access to the contents

2. Next, go to your DNS provider to edit your DNS settings. My domain is held by easyCgi.com so I used their 'DomainCentral' portal to edit my DNS

3. Add a CNAME alias to point your new subdomain to the s3 bucket name


4. Wait a while for the DNS system to update itself and if all is well, you should be able to access your bucket resources directly, e.g.: http://assets.eoinclayton.net/GreenTick.jpg

Thursday, May 8, 2014

HTML5 AppCache with MVC bundles


HTML5 has many handy additions and I recently had the opportunity to use the Appcache Manifest. This feature enables your site to work offline which is particularly useful for mobile orientated sites as users may not always have a reliable 'net connection.

The following guide goes through the approach I took for getting it working via .Net's MVC framework.

Prep the HTML tag

On the page you want to have a manifest for, add the following:

<html lang="en" manifest="/AppCache/AppManifest">

This will tell the browser to look for the appcache manifest.

Setup the Model

We don't need many properties on our Model for the manifest generation. The following will do
public class AppCacheModel
    {
        public string AssemblyVersion { get; set; }
        public List< string> CacheCollection { get; set; }
    }
  • An assembly version which will help update the manifest file contents after every build
  • A CacheCollection list of urls to add to the cache

Add a view

You will need to set various headers to get the manifest file to work correctly. In particular, set the MIME content type to "text/cache-manifest" and set the cache headers so that the manifest never caches.
@model MyWebsite.Models. AppCacheModel
@{
    Layout = null;
    Response.ContentType = "text/cache-manifest";
    Response.Cache.SetCacheability( HttpCacheability.NoCache);
    Response.Cache.SetExpires( DateTime.MinValue);
}CACHE MANIFEST
# Server Assembly Version: @ Model.AssemblyVersion
NETWORK:
*
CACHE:
@foreach (string cacheItem in Model.CacheCollection)
{
    @ cacheItem
}

Instead of hard-coding the individual assets in the Cache section, we are going to get the controller to add them in for us...

Code up the Controller

The controller populates the model with:
  • the assembly version
  • the links to the various assets to be cached. 
The code handles MVC Bundles and also generates a list of asset urls from target folders. If the assets are in a CDN bucket somewhere, you can also provide the link to the CDN.

public class AppCacheController : Controller
    {
        public ActionResult AppManifest()
        {
            // Build the model
            var model = new AppCacheModel();

            model.AssemblyVersion = GetType().Assembly.GetName().Version.ToString();
            model.CacheCollection = new List< string>();
            model.CacheCollection.Add(WriteBundle( "~/bundles/jquery"));
            model.CacheCollection.Add(WriteBundle( "~/bundles/site"));
            model.CacheCollection.Add(GetPhysicalFilesToCache( "~/images", "*.jpg" , "http://cdn.eoinclayton.net/website" ));
            model.CacheCollection.Add(GetPhysicalFilesToCache( "~/images", "*.png" , "http://cdn.eoinclayton.net/website" ));
            model.CacheCollection.Add(GetPhysicalFilesToCache( "~/css", "style*.css", string .Empty));

            return View(model);
        }

        private string WriteBundle( string virtualPath)
        {
            var bundleString = new StringBuilder();
            bundleString.AppendLine( Scripts.Url(virtualPath).ToString());
            return bundleString.ToString();
        }

        private string GetPhysicalFilesToCache( string relativeFolderToAssets, string fileTypes, string cdnBucket)
        {
            var outputString = new StringBuilder();
            var folder = new DirectoryInfo(Server.MapPath(relativeFolderToAssets));
            foreach ( FileInfo file in folder.GetFiles(fileTypes))
            {
                string location = ! String.IsNullOrEmpty(cdnBucket) ? cdnBucket : relativeFolderToAssets;
                string outputFileName = (location + "/" + file).Replace("~" , string.Empty);
                outputString.AppendLine(outputFileName);
            }
            return outputString.ToString();
        }
    }


How to test

1. Open your app/site in Firefox
2. Once it is loaded, close down Firefox
3. Re-open and set Firefox to Offline mode (File -> Offline)
4. Hit your app/site. Even though you are offline, it should still work

You can follow a similar approach on your device, just use Airplane mode instead of Offline mode.

Things to watch out for

  1. Do NOT cache the manifest file! Just don't do it. If you do, your site/app may never update for your clients...!
  2. Refrain from using the "No-Store" response header on the manifest file. The offline site continued to work as expected in Chrome, but in Firefox the entire site stopped working offline...
  3. Use Chrome's Javascript Console (Tools -> Javascript Console) to see if the manifest is being loaded correctly. It will give a breakdown for each item being cached and will tell you, for example, if you try to cache a non-existing asset
  4. To deactivate the appcache manifest, just removing it from the HTML tag is not enough. My site stopped updating entirely when I did this. You will also need to remove the manifest url/view.
  5. You do have a additional bucket to store data known as the 'Local Storage' (http://diveintohtml5.info/storage.html)
  6. There are also optional Javascript events to listen to. This handy Stackoverflow post goes through what the different events do: http://stackoverflow.com/a/20045644

Wednesday, May 7, 2014

Setting up SourceTree, BitBucket, SSH with a proxy on Windows

When using SourceTree to connect to Bitbucket with an SSH key through a proxy server, there are a number of steps to follow...

SourceTree setup

1. Open SourceTree, goto Tools and select 'Create or Import SSH Keys'


2. A dialog will open up. Click Generate to generate a public private key pair:
 
  • Provide a suitable passphrase and save both the public and private keys

3. In your system tray, right click on Pageant (PuTTY authentication agent) and select "Add Key"
 
  • Select the private key you created in step 2

Proxy Setup with Putty

1. Setup your HTTP Proxy settings in Putty
  • Click on your Start button and search for Putty
  • When Putty comes up goto the Session Category 'Default Settings' and then Load
  • Goto the Proxy category and add in the proxy settings, making sure to select the right proxy type
  • Go back to the Session Category and save the updates to the Default Settings
 

Bitbucket setup

1. Goto BitBucket, navigate to "Your Account"
2. Under SSH Keys, add the Public Key saved above
 
3. Restart SourceTree and then add a new repo with the ssh url for your repo
To find the right URL to use, navigate to your repo in BitBucket and you'll see the SSH url on the right:
 

Monday, November 4, 2013

Using Xamarin Sharpie to build a Flurry assembly

The main project I work on is a series of Xamarin based apps for iOS devices.
To help track usage stats we use Flurry Analytics which gives us a handy dashboard showing us session usage, device & OS breakdown, error logs, etc.

With the iOS 7 release, Flurry required a update:







Flurry doesn't have a .Net assembly out-of-the-box, so we needed a way of converting the native library into .Net.

When we had added Flurry to our project ages ago, we started with the Flurry monotouch-bindings to get things started. However this time Flurry had changed something in their library and the monotouch-binding no longer worked correctly. Trying to run the app with the built assembly resulted in build errors including:
Undefined symbols for architecture armv7:
  "_SecItemAdd", referenced from:
      -[FlurryKeychainWrapper setData:forKey:] in libFlurry.a(libFlurry.a-armv7-master.o)
  "_SecItemCopyMatching", referenced from:
      -[FlurryKeychainWrapper dataForKey:] in libFlurry.a(libFlurry.a-armv7-master.o)
  "_SecItemDelete", referenced from:
      -[FlurryKeychainWrapper removeObjectForKey:] in libFlurry.a(libFlurry.a-armv7-master.o)

Missing "sec" stuff. Doh!
I'll walk you through what I had to do to sort this out.

What's changed in Flurry?

Before starting, I had a quick look at the release notes to see if anything jumped out:














Hmmm…. point 4…. a new "Security" framework was added. Best keep that in mind for when we do the linking later.

Building a Flurry assembly

Firstly, download the native SDK from Flurry. You will end up with the following files as part of the sdk:






Next, we will need to convert the ".a" native library into a .Net assembly. Head over to Xamarin and download Sharpie. This useful tool will parse the Flurry.h header file and output a C# binding definition.















Follow through the steps in the tool and you'll end up with a binding file which will include something similar to:
using System;
using MonoTouch.Foundation;

namespace FlurryAnalytics {

     public enum FlurryLogLevel : [unmapped: unexposed: Elaborated] {
          None = 0,
          CriticalOnly,
          Debug,
          All
     }
     [BaseType (typeof (NSObject))]
     public partial interface Flurry {

          [Static, Export ("appVersion")]
          string AppVersion { set; }

          [Static, Export ("getFlurryAgentVersion")]
          string GetFlurryAgentVersion { get; }

          [Static, Export ("showErrorInLogEnabled")]
          bool ShowErrorInLogEnabled { set; }

          [Static, Export ("debugLogEnabled")]
          bool DebugLogEnabled { set; }
        
          [Static, Export ("sessionContinueSeconds")]
          int SessionContinueSeconds { set; }

          [Static, Export ("secureTransportEnabled")]
          bool SecureTransportEnabled { set; }
          ….

Its done a pretty good job, but has just gotten a bit confused with the enum 'FlurryLogLevel'. I took that enum out of the binding file and added it to a new "extras.cs" file I created:
namespace FlurryAnalytics
{
     public enum FlurryLogLevel
     {
          None = 0,
          CriticalOnly,
          Debug,
          All
     }
}

I now had the following four files in the folder:






Next, we will need to give some build instructions to Xamarin to tell it how to handle building and linking the native library and the binding definition. To do that, we add an AssemblyInfo.cs with the following content:
using System;
using MonoTouch.ObjCRuntime;

[assembly: LinkWith ("libFlurry_4.2.3.a", LinkTarget.Simulator | LinkTarget.ArmV7, Frameworks = "SystemConfiguration Security", ForceLoad = true)]

The linker is going to be told to grab the libFlurry_4.2.3.a library, link in the simulator and armv7 targets (armv7 is for the devices) and include the SystemConfiguration framework. Notice also that we included the Security framework. That should take care of the build errors.

Lastly, lets take the monotouch-bindings 'makefile' and alter it to better match our needs:
BTOUCH=/Developer/MonoTouch/usr/bin/btouch

all: FlurryAnalytics.dll

FlurryAnalytics.dll: Makefile AssemblyInfo.cs FlurryAnalytics.cs libFlurry_4.2.3.a
     $(BTOUCH)  FlurryAnalytics.cs AssemblyInfo.cs extras.cs --out=$@ --link-with=libFlurry_4.2.3.a,libFlurry_4.2.3.a

You'll now have the following files:








Open up a terminal, go to the above folder and run the build by typing "make". A FlurryAnalytics.dll should appear in the folder. You can include this in your project and should be able to start using the Flurry magic.

Wednesday, October 30, 2013

Intro to using the Windows Debugger

Windows Debugger can be downloaded as part of the Windows SDK. I would install both the 32 and 64bit versions.

I used it in the past when diagnosing why a website was performing poorly during peak periods. Users would hit the website url, wait for 30+ seconds and then the site would appear quickly. Even the home page which did nothing (i.e. no db hits), was slow to appear.

I got a memory dump from the server and using the analysis approach below could see that all available IIS worker threads were in use and all were busy rendering. The server was starved of threads and user requests were ending up in a queue waiting to be processed.

I tweaked the IIS settings and added more worker threads. The problem went away. Thanks Windows Debugger!

How do you to get a memory dump?

1. Open a command prompt
2. Get the list of worker process ids:
  • %systemroot%\system32\inetsrv\APPCMD list wps
    • Your worker process should be in the list
3. Change to the Debugging Tools directory. On my machine that was at:
  • cd C:\Program Files (x86)\Windows Kits\8.0\Debuggers\x86 4. Create a directory for the memory dump. I went with c:\dumps
5. Run the following, substituting your process id for xxxx:
  • ADPlus -Hang -p xxxx -quiet -o c:\dumps. You'll end up with a .dmp file in the c:\dumps folder.

Opening the memory dump in WinDBG

1. Run WinDBG. You'll see something similar to the following:

2. Goto File -> Open Crash Dump. You should then see something similar to the following:

3. Set the symbol path:
  • .sympath srv*c:\Symbols*http://msdl.microsoft.com/download/symbols
  • .symfix
  • .reload 

4. Load the .Net CLR
  • .Net 3.5: .loadby sos mscorwks
  • .. or for .Net 4.0: .loadby sos clr

Working out what each of the threads are doing

1. Get the list of threads in the memory dump
  • !Threads -special 

2. Switch to a ThreadPoolWorker thread, e.g. thread 11:
  • ~11s 

3. Get the stacktrace for this thread
  • !clrstack 

Using the stacktrace, you can then see what the thread is doing. Switch then to a different thread to see what it is doing. Using this approach I could work out that all threads were busy doing normal activities and that the website just needed more worker threads.

MVC HTTP Headers for CDN caching

On a recent project I added upgraded our systems to use a Content Delivery Network (CDN). The project supplied content to users and we wanted to offload the 'heavy-lifting' of the content supply away from our PROD server onto a more robust delivery network.

The mobile clients would request the content from the CDN. If the CDN hadn't cached the content yet, it would grab it from our PROD server, cache it, and respond back to the clients. For this caching to work, though, some HTTP Header magic was required.

Which headers should you watch out for?

  • Cache control should be public
    • For the CDN to cache the content on behalf of your origin server, set the Cache-control=public
  • Expires Date
    • To stop the CDN regularly requested the same static content from your PROD server, set the Expires-Date setting to be in the far future (e.g. > 1 year)
  •  Vary 
    • Watch out for the 'Vary' header. MVC3 had initially defaulted to adding the "vary=*" http header which stopped the CDN from caching the data. 

A useful RFC to refer to is: RFC 2616 13.6:
A Vary header field-value of * always fails to match and subsequent requests on that resource can only be properly interpreted by the origin server.

.. or if you prefer the MSDN page: http://msdn.microsoft.com/en-us/library/system.web.httpcachepolicy.setomitvarystar.aspx

If you set Vary="NONE", this gets outputted as "Vary=*", which isn't very useful. Either get rid of the Vary header or set it to something broad such as "Vary: Accept-Encoding"

How to implement this in MVC?

[OutputCache(Location = OutputCacheLocation.Any, Duration = 31536000, VaryByParam = "")]
public ActionResult GetMyCDNContent(string someSortOfIdenfier)
{...
OutputCacheLocation.Any will be outputted as: Cache-Control: public
Duration = 31536000 will be outputted as:
  • Cache-Control: max-age=31536000
  • Expires: {a date a year from today}
  • VaryByParam="" will not output any Vary= header
  • This is what we needed; the content was going to be the same for all clients.

Here's how it ends up looking in Fiddler:






You might want something different if you have alternate caching needs for different encodings, such as support for gzip and non-gzip caching:
[OutputCache(Location = OutputCacheLocation.Any, Duration = 31536000, VaryByParam = "", VaryByHeader = "Accept-Encoding" )]
public ActionResult GetMyCDNContent(string someSortOfIdenfier)
{...

Here's how it ends up looking in Fiddler: