Abusing UIKit for Gaming in Xamarin.iOS, Part 3: Playing Sounds

This is the third in a [glacially-paced] series of Abusing UIKit blog posts giving some background on the development that want into producing Smudges, a simple game written entirely in Xamarin.iOS where fun shapes in various colors show up on the screen wherever a tap is detected. It was original created to give my two-year-old something fun to play while going tap-crazy on the screen. The game evolved from those “play-testing” sessions. If you have your own little ones and want something fun to distract them, Smudges is availabe on the App Store. At this point, I plan to continue adding features to it as I can. Let me know what you think about Smudges, or these blog posts, in the comments below or find @patridgedev on Twitter.

It's hard to show sound with a picture. This is where the sounds go when you play them. :P

Let There Be Noise!

Playing sounds in your apps can make for some great user interactions. Of course, it can also be used for far more annoying uses. For Smudges, each time an icon is placed on the screen with a tap, a random sound is played from a set of noises originally generated via as3sfxr (think Atari-era synth sounds).

As a disclaimer, this will only cover playing the random sound here and there. If you want timed, scheduled, or synchronized sounds, you will want to spend some time getting to know Audio Queue Services.

Getting Sounds Ready

Apple suggests different audio formats to meet different needs. Your project needs may vary, but if you aren’t doing crazy high-quality music tracks, there is a lot of flexibility.

You can play WAV, MP3, and CAF files. I wasn’t able to play OGG files directly, but there are lots of tools to convert between the various formats. I tend to use VLC to convert between various formats. At the risk of turning this into a tutorial about converting audio files, open VLC and go to File > “Convert / Stream…”.

  1. Select the source file
  2. Choose a profile to determine your output format
  3. Select “Save as File” and choose an output location

VLC audio format conversion example settings.

Sound, Meet Project. Project, Meet Sound

Whatever sounds and audio format you decide to use, add your sounds to the Resources folder in your iOS project. Feel free to use a subfolder to organize things however you prefer. Ultimately, you will end up with a file added to your project with a build action of “BundleResource”, ensuring it is set up for use at runtime.

Sound files added as project resources.

With your sounds in place, it’s just a few lines of code until you can start sending vibrations to your users’ ears.

Playing the Sounds

For these simple UI-style sounds, I find it best to throw an AVAudioPlayer at the problem. You spin one up with an NSUrl pointing to the resource, and tell it to play when you need it.

using AVFoundation;
...
var fileUrl = new NSUrl(NSBundle.MainBundle.PathForResource("Sounds/wubwub", "caf"), false);
var player = AVAudioPlayer.FromUrl(fileUrl);
...
player.Play();

If you want to help ensure your sound plays back immediately when asked to do so, you can tell the player to get itself ready to go.

player.PrepareToPlay();

For example, you can build up your audio player and immediately call this method in your ViewDidLoad method. Then, when your sound plays later, say, in response to an event handler firing, it is theoretically ready to go. The Apple AVAudioPlayer docs on this method don’t go into detail too much, but presumably this consumes memory and audio hardware resources. You will want to weigh the trade-offs in your playback based on the size/format of your sounds and how frequently you need each one.

When You’re Done…

If you want to run some code after your sound plays, AVAudioPlayer.FinishedPlaying is the event you’ll need.

player.FinishedPlaying += doSomethingCoolAfterSoundPlays;

In the sample code, I just throw a rotation animation at the button that triggered the sound. In the sample code, I just inline an event handler that rotates the button that triggered the sound. If you have a bunch of audio players, though, you could easily tie them all together to a common handler to save on lines of code.

Best Practices?

I’m not one to shy away from acknowledging when I don’t know something, and this is one of those times. When it comes to the best practices for audio playback, I only know what has worked well for me so far. A lot of the detailed information you find about playing audio on iOS is in game engines rather than hitting the core APIs directly. There is probably some value in digging into how these engines do their dirty work, under the hood.

Until then, I’ve gotten by pretty well with simple audio playback using the above technique. Smudges allows for simultaneous taps from 10 fingers (on iPads, iPhones only allow 5). When a pair of kids goes tap-crazy on Smudges all they can, it seems to hold up quite well, if not a little muddled. I have all my players in a list, and just grab a random one to play; that’s how the sample code does it as well. Take that for what it’s worth. If you find a better/safer way to do things, please do share in a comment below.

One last shameless plug…

Find Smudges on the App Store, available for iPhone, iPad, or iPod Touch. It’s especially fun on iPad where the hardware allows for ten simultaneous touch points.

Abusing UIKit for Gaming in Xamarin.iOS, Part 2: Using Custom Fonts

This is the second in a series of Abusing UIKit blog posts giving some background on the development that want into producing Smudges, a simple game written entirely in Xamarin.iOS where fun shapes in various colors show up on the screen wherever a tap is detected. It was original created to give my two-year-old something fun to play while going tap-crazy on the screen. The game evolved from those “play-testing” sessions. If you have your own little ones and want something fun to distract them, Smudges is availabe on the App Store. At this point, I plan to continue adding features to it as I can. Let me know what you think about Smudges, or these blog posts, in the comments below or find @patridgedev on Twitter.

Demo of using a custom icon font in a UILabel and adjusting the size

Using an Icon Font

Using an icon font can be great for a typical app for substituting a mess of PNGs. For icons, the size savings is probably minimal, but dealing with a single font file compared to a folder of icon images in numerous DPI variations can be much nicer. Since we are dealing with “plain” text in a label, color is controlled by manipulating the label’s text color. For Smudges, the icon font was all about having something visually enjoyable pop onto the screen beyond a basic rectangular view. Fortunately, most icon fonts have a few characters that are just for fun.

Font, Meet Project. Project, Meet Font

Whether you are using a custom icon font or a regular font in your project, adding it is exactly the same. First, pick your favorite TTF font file. Either drag it onto the Resources folder in Xamarin Studio or right-click to “Add Files…” there. You can also put it in a subfolder within Resources (e.g., “Fonts”) as I did for this project, since later demos may have fonts and sounds.

Once you have a font in your project, there are two more steps to make it work: setting up Info.plist and setting the build action. Since you are already in the solution view, right-click the new font and get its properties. Change the “Build action” to “BundleResource”. Now you need to tell the app that it needs to load custom fonts and which files will be provided by adding some details to Info.plist. Open your projects Info.plist file and change to Source view with the bottom tab choices. Add a new root entry and pick “Fonts provided by application” from the choices. This will create a new array where you will put any file names of the custom fonts within your application bundle. In this case, I added Fonts/fontawesome-webfont.ttf to the array.

Custom fonts under Fonts provided by application in Info.plist

With the file bundled and Info.plist all prepped, it’s just a matter of creating a UIFont with the correct font name.

UIFont IconFont = UIFont.FromName("FontAwesome", CharacterSize);

Getting the Font Name

You may not always know exactly what your font’s name is as seen by iOS. The simple way to find it is to open it with the baked-in OS X Font Book application. The font’s name will be shown as the window title.

Getting the font's name with Font Book

One other method is to run your app and pull all the available font names and search for the one you think is right for the newly embedded font. (This has the added benefit of verifying you have all the previous steps set up correctly; if the font shows up in this list, the app found it and loaded it.)

// Dump out an alphabetical list of all available font names on the device.
var availableFontNames = UIFont.FamilyNames.SelectMany(familyName => UIFont.FontNamesForFamilyName(familyName)).ToList().OrderBy(fontName => fontName);
Console.WriteLine(string.Join("\n", availableFontNames));

Quite a few fonts are already ready to go for your apps to use in iOS. Given that, you may have trouble finding the name of your new font in that large list. It’s definitely worth trying the Font Book approach before you verify your project is configured correctly using this approach.

Debugging to get font names available to your app

Determining character codes

Once you figure out what characters from your font that you want to use, you will need the hexadecimal character codes to use them as strings. While I do tend to look for simple solutions to problems, I haven’t found a simple solution to determining the character codes for a font yet. I often do a silly dance of installing the font in Windows and viewing it within the Character Map application.

Looking up character codes in the Windows Character Map

Since font icons often use the Unicode private use space, you may have to scroll around to find the characters you want to use. Once you find the character you want to use, write down its character code, F0FB in the case of the fighter jet in the screenshot.

Sometimes you get lucky with an icon font. Font Awesome has a handy listing of icons on their site. If you use your browser’s development tools to inspect the desired icon, you can usually find the hexadecimal code in the CSS.

Inspecting CSS of desired character use on the web

Once you find the appropriate CSS rule, grab the hexadecimal character code.

Looking up character codes in browser CSS rule

From there, take the character code and use it as any normal char (or string) in your C# code.

static readonly UIFont IconFont = UIFont.FromName("FontAwesome", 40f);
//...
char jet = '\uf0fb';
var label = new UILabel(RectangleF.Empty) {
    Font = IconFont,
    Text = jet.ToString(),
    TextColor= someForeColor,
    BackgroundColor = someBackgroundColor,
};
label.SizeToFit();
Add(label);

If you happen to see an unusual “W”-looking character, you probably forgot to set it to use your icon font.

Missing font character fail

Adjusting Your Icon Size

Since your new icons aren’t plain images with pre-determined sizes, you need to figure out the size you want to use. If you can get away with it, figure out your font point size beforehand and roll with it as we did above. If you sizing requirements are more fluid, you will have to put a little more effort into it.

I’ve tried a few different approaches trying to get flexible sizing to work. First, I tried the direct route: set the font size overly large and toggle AdjustsFontSizeToFitWidth to have the system do the hard work. There may be a way to get this to work, but I had less-than-ideal results with that approach.

Icon font sizing fail with AdjustsFontSizeToFitWidth

At some point in playing with iOS, you will probably need to know the size of a string before it ends up on the screen. This is where the various overloads for NSString.StringSize come in. There is even an overload that appears to be designed to return the maximum font size for a given width, but I couldn’t get it to return a useful value. Instead, I created a hack function that calculated the largest font size by trying incrementally larger sizes until StringSize came out too large.

// CAUTION: total hack!
public static float GetMaxFontSize(this string source, UIFont font, SizeF sizeRestriction) {
    float maxFontSize = font.PointSize;
    SizeF latest = SizeF.Empty;
    using (NSString nssDescriptionWithoutHtml = new NSString(source.ToString())) {
        while (latest.Width < sizeRestriction.Width && latest.Height < sizeRestriction.Height) {
            latest = nssDescriptionWithoutHtml.StringSize(font.WithSize(maxFontSize), sizeRestriction.Width, UILineBreakMode.Clip);
            if (latest.Width < sizeRestriction.Width && latest.Height < sizeRestriction.Height) {
                maxFontSize += 0.1f;
            }
        }
    }
    return maxFontSize;
}

Using a font size calculation to determine the largest possible size for a given area

Demo

In the demo, you’ll see a UILabel used with a custom icon font (the jet icon seen here). At the bottom is a slider that controls the size of the label, showing how icon fonts can make for sharp icons at any size. As well, a UIButton with a “refresh” icon is there. When tapped, it spins the jet icon using the rotate animation code from one of my first Xamarin.iOS blog posts.

Demo Source Code

For more details on the font-sizing code or any of the rest of this post’s code, just pop over to the Abusing UIKit for Gaming GitHub repo. The custom font code project has been added to this project, and I’ll be adding new projects to that solution as I add posts to this series.

Picking a font, making your own icon font, resources

When it comes time to finding your own icon font, there are a lot of choices. Font Awesome is nice, and the price and license is nice (read: free). If it meets your needs, you are good to go. If you need something different, though, there are a lot of great icon fonts out there to use.

If you already have your own vector assets, there is also the choice of making your own icon font. Trello switched to an icon font at one point. They even put out a nice blog post about it.

One last shameless plug…

Find Smudges on the App Store, available for iPhone, iPad, or iPod Touch. It’s especially fun on iPad where the hardware allows for ten simultaneous touch points.

Abusing UIKit for Gaming in Xamarin.iOS, Part 1: Detecting Taps and Placing Views with UITapGestureRecognizer

This is the first in a series of Abusing UIKit blog posts giving some background on the development that want into producing Smudges, a simple game written entirely in Xamarin.iOS where fun shapes in various colors show up on the screen wherever a tap is detected. It was original created to give my two-year-old something fun to play while going tap-crazy on the screen. The game evolved from those “play-testing” sessions. If you have your own little ones and want something fun to distract them, Smudges is availabe on the App Store. At this point, I plan to continue adding features to it as I can. Let me know what you think about Smudges, or these blog posts, in the comments below or find @patridgedev on Twitter.

Demo animation of placing views with a UITapGestureRecognizer.

Where Did They Touch?

Smudges has a simple game mechanic: tap the screen, new shape appears. The first step is figuring out when and where a tap occurred. The simple approach is to put a UIButton where you need to detect a touch, attaching a handler to its TouchUpInside event.

UIButton someButton = new UIButton(someFrameRectangle);
AddSubview(someButton);
someButton.TouchUpInside += (sender, e) => {
    Debug.WriteLine("Touched somewhere on this button.");
};

Of course, while incredibly simple, this is only practical for interface elements; it’s not so great for doing something at the exact location of a tap. To know the X-Y location where the user’s finger is touching the screen, you need to wander into UITapGestureRecognizer territory (or beyond).

UITapGestureRecognizer tapRecognizer = new UITapGestureRecognizer() {
    NumberOfTapsRequired = 1, // default is 1, adjust as needed
};
tapRecognizer.AddTarget((sender) => {
    PointF location = tapRecognizer.LocationOfTouch(0, MainView);
    Debug.WriteLine("Touched: {0}", location);
});
someContainerView.AddGestureRecognizer(tapRecognizer);

Simple enough as well, and a great choice for double-taps…and, I suppose, three-finger triple-taps, if you want to get crazy.

Placing the View

Once you have the tap location, doing something with it is fairly trivial.

tapRecognizer.AddTarget((sender) => {
    PointF location = tapRecognizer.LocationOfTouch(0, MainView);
    UIView newView = new UIView(new RectangleF(location, new SizeF(50f, 50f))) {
        BackgroundColor = UIColor.Red,
    };
    someContainerView.AddView(newView);
});

This will happily place a 50×50 rectangle on the screen with its top-left corner sitting right where the tap was detected. Centering on that location was my goal, which is easy enough by setting the new view’s Center instead of a full frame.

tapRecognizer.AddTarget((sender) => {
    PointF location = tapRecognizer.LocationOfTouch(0, MainView);
    UIView newView = new UIView() {
        Size = new SizeF(50f, 50f),
        Center = location,
        BackgroundColor = UIColor.Red,
    };
    someContainerView.AddView(newView);
});

Make it Pretty

One thing I tend to do in demos of any kind is introduce some randomness to the mix. It keeps things much more interesting to everyone, especially me while I am running it for the thousandth time to get everything just right. In this case, let’s pick a random color for each one.

static Random rand = new Random();
public static UIColor GetRandomColor() {
    int red = rand.Next(255);
    int green = rand.Next(255);
    int blue = rand.Next(255);
    UIColor color = UIColor.FromRGBA(
        (red / 255.0f),
        (green / 255.0f),
        (blue / 255.0f),
        1f);
    return color;
}

If you need random numbers that hold up over time, you will want to investigate better random number generators, but System.Random serves its purpose well for these uses. Throwing all that into the mix, we now get a new “fun” gesture recognizer target that places a rectangle of a random color centered right at the touch location.

UITapGestureRecognizer tapRecognizer = new UITapGestureRecognizer() {
    NumberOfTapsRequired = 1
};
tapRecognizer.AddTarget((sender) => {
    PointF location = tapRecognizer.LocationOfTouch(0, MainView);
    UIView newView = new UIView() {
        Size = new SizeF(50f, 50f),
        Center = location,
        BackgroundColor = GetRandomColor(),
    };
    someContainerView.AddView(newView);
});
someContainerView.AddGestureRecognizer(tapRecognizer);

Caution: Memory Leak Ahead

When you go adding views all the time, with enough time, you are bound to run up against memory pressures. The simplest solution is to keep track of added views so you can remove them later. In this demo, I simply schedule a task to do so after some time elapses. In a later demo, I will have a simple demo using a Queue that does just that. Another option is a pool of views, something many game engines offer directly. There’s lots of room for turning this potential limitation into something fun.

Beyond UITapGestureRecognizer

This version allows the detection of a single finger tapping. If you use two fingers to tap at the same time, nothing will happen. In a later post, I will address recognizing simultaneous touches.

Demo Source Code

For details on the view queueing system or any of the rest of this post’s code, just pop over to the GitHub repo for the Abusing UIKit for Gaming in Xamarin.IOS series. I’ll be adding new projects to that solution as I add posts to this series.

One last shameless plug…

Find Smudges on the App Store, available for iPhone, iPad, or iPod Touch. It’s especially fun on iPad where the hardware allows for ten simultaneous touch points.