Testing completion handler APIs with Swift Testing

The Swift testing framework is an incredibly useful tool that allows us to write more expressive tests with convenient and modern APIs.

This is my first post about Swift Testing, and I’m mainly writing it because I wanted to write about something that I encountered not too long ago when I tried to use Swift testing on a code base where I had both async code as well as older completion handler based code.

The async code was very easy to test due to how Swift Testing is designed, and I will be writing more about that in the future.

The completion handler base code was a little bit harder to test, mainly because I was converting my code from XCTest with test expectations to whatever the equivalent would be in Swift testing.

Understanding the problem

When I started learning Swift testing, I actually looked at Apple's migration document and I found that there is an something that’s supposed to be analogous to the expectation object, which is the confirmation object. The examples from Apple have one little caveat in there.

The Swift Testing example looks a little bit like this:

// After
struct FoodTruckTests {
  @Test func truckEvents() async {
    await confirmation("…") { soldFood in
      FoodTruck.shared.eventHandler = { event in
        if case .soldFood = event {
          soldFood()
        }
      }
      await Customer().buy(.soup)
    }
    ...
  }
  ...
}

Now, as you can see in the code above, the example that Apple has shows that we have a function and a call to the confirmation function in there, which is how we’re supposed to test our async code.

They call their old completion handler based API and in the event handler closure they call their confirmation closure (called soldFood in the example).

After calling setting the event handler they await Customer().buy(.soup).

And this is really where Apple wants us to pay close attention because in the migration document, they mention that we want to catch an event that happens during some asynchronous process.

The await that they have as the final line of that confirmation closure is really the key part of how we should be using confirmation.

When I tried to migrate my completion handler based code that I tested with XCTestExpectation, I didn't have anything to await. My original testing code looked a little bit like this:

func test_valueChangedClosure() {
  let expect = expectation(description: "Expected synchronizer to complete")

  let synchronizer = Synchronizer()
  synchronizer.onComplete = {
    XCTAssert(synchronizer.newsItems.count == 2)
    expect.fulfill()
  }

  synchronizer.synchronize()
  waitForExpectations(timeout: 1)
}

Based on the migration guide and skimming the examples I though that the following code would be the Swift Testing equivalent:

@Test func valueChangedClosure() async {    
  await confirmation("Synchronizer completes") { @MainActor confirm in
    synchronizer.onComplete = {
      #expect(synchronizer.newsItems.count == 2)
      confirm()
    }

    synchronizer.synchronize()
  }
}

My code ended up looking quite similar to Apple’s code but the key difference is the last line in my confirmation. I’m not awaiting anything.

The result when running this is always a failing test. The test is not waiting for me to call the confirm closure at all. That await right at the end in Apple’s sample code is pretty much needed for this API to be usable as a replacement of your expectations.

What Apple says in the migration guide when you carefully read is actually that all of the confirmations have to be called before your closure returns:

Confirmations function similarly to the expectations API of XCTest,
however, they don’t block or suspend the caller while waiting for a
condition to be fulfilled. Instead, the requirement is expected to be confirmed (the equivalent of fulfilling an expectation) before confirmation() returns

So whenever that confirmation closure returns, Swift Testing expects that we have confirmed all of our confirmations. In a traditional completion handler-based setup, this won't be the case because you're not awaiting anything because you don't have anything to await.

This was quite tricky to figure out.

Write a test for completion handler code

The solution here is to not use a confirmation object here because what I thought would happen, is that the confirmation would act a little bit like a continuation in the sense that the Swift test would wait for me to call that confirmation.

This is not the case.

So what I've really found is that the best way to test your completion handler-based APIs is to use continuations.

You can use a continuation to wrap your call to the completion handler-based API and then in the completion handler, do all of your assertions and resume your continuation. This will then resume your test and it will complete your test.

Here’s what that looks like as an example:

@Test func valueChangedClosure() async {
    await withCheckedContinuation { continuation in
        synchronizer.onComplete = {
            #expect(synchronizer.newsItems.count == 2)
            continuation.resume()
        }

        synchronizer.synchronize()
    }
}

This approach works very well for what I needed, and it allows me to suspend the test while my callback based code is running.

It's the simplest approach I could come up with, which is usually a good sign. But if you have any other approaches that you prefer, I would love to hear about them, especially when it relates to testing completion handler APIs. I know this is not a full-on replacement for everything that we can do with expectations, but for the completion handler case, I think it's a pretty good replacement.

When not to use continuations for testing completion handler code

The approach of testing outlined above assumes that our code is somewhat free of certain bugs where the completion handler is never called. Our continuation doesn't do anything to prevent our test from hanging forever which could (let's be honest, will) be an issue for certain scenarios.

There are code snippets out there that will get you the ability to handle timeouts, like the one found in this gist that was shared with me by Alejandro Ramirez.

I haven't done extensive testing with this snippet yet but a couple of initial tests look good to me.