Preventing unwanted fetches when using NSFetchedResultsController and fetchBatchSize

This article covers a topic that is extensively covered in my Practical Core Data book. This book is intended to help you learn Core Data from scratch using modern techniques and every chapter features sample apps in SwiftUI as well as UIKit whenever this is relevant.

When you use Core Data in a UIKit or SwiftUI app, the easiest way to do this is through a fetched results controller. In SwiftUI, fetched results controller is best used through the @FetchRequest property wrapper. In UIKit, a fetched results controller can be conveniently set up to provide diffable data source snapshots for your table- or collection view while SwiftUI's @FetchRequest will conveniently update your UI as needed without requiring any extra work.

If you're somewhat knowledgable in the realm of Core Data, you've heard about the fetchBatchSize property.

This property is used to fetch your data in batches to prevent having to fetch your entire result set in one go. When you're dealing with a large data set, this can be a huge win.

However, when you're using a fetched results controller with diffable data sources and you set a fetchBatchSize, you'll find that your fetched results controller will initially fetch all of your data using your specified batch size. In other words, your data will be retrieved immediately using many small fetches. Once you start scrolling through your list, the fetched results controller will fetch your data again. using the specified batch size

Because SwiftUI's @FetchRequest is built on top of NSFetchedResultsController, you'll see the exact same problem manifest in a SwiftUI app that uses @FetchRequest with a fetch request that has its fetchBatchSize set.

In this post, I will briefly explain what the problem is exactly, and I'll show you a solution for a UIKit solution. A solution for SwiftUI will be published in a seperate post.

Understanding the problem

The easiest way to spot a problem like the one I described in the introduction of this post is through Core Data's debug launch arguments so you can see the SQLite statements that Core Data runs to retrieve and save data.

When you enable these launch arguments in an app that uses fetchBatchSize combined with a fetched results controller that provides diffable data source snapshots, you'll notice the following:

1. First, all objectIDs are fetched in the correct order so the fetched results controller (or @FetchRequest which uses a fetched results controller under the hood as far as I can tell) knows the number of items in the result set, and so it knows how to page requests.
2. Then, all managed objects are fetched in batches that match the batch size you've set.
3. Lastly, your managed objects are fetched in batches that match your batch size as you scroll through your list.

The second point on this list is worrying. Why does a fetched results controller fetch all data when we expect it to only fetch the first batch? After all, you set a batch size so you don't fetch all data in one go. And now your fetched results controller doesn't just fetch all data up front, it does so in many small batches.

That can't be right, can it?

As it turns out, it seems related to how NSFetchedResultsController constructs a diffable data source snapshot.

I'm not sure how it works exactly, but I am sure that generating the diffable data source snapshot is what triggers these unwanted fetch requests. If a UIKit app, you can quickly verify this by commenting out your NSFetchedResultsControllerDelegate's controller(_:didChangeContentWith:) method. One you do this, you'll notice that your fetched results controller no longer fetches all data.

So how can you work around this?

As it turns out, there's no straightforward way to do this. The best way I've found is to stop using diffable data sources completely and instead use the older delegate methods from NSFetchedResultsControllerDelegate to update your table- or collection view.

In the next section, I'll show you how you can implement the appropriate delegate methods and update an existing collection view. How you build the collection view is up to you, as long as you populate your collection view by implementing the UICollectionViewDataSource methods rather than using diffable data sources.

Preventing unwanted requests in a UIKit app

The easiest way to prevent unwanted requests in a UIKit app is to get rid of the controller(_:didChangeContentWith:) delegate method that's used to have your fetched results controller construct diffable data source snapshots. Instead, you'll want to implement the following four NSFetchedResultsControllerDelegate methods:

• controllerWillChangeContent(_:)
• controller(_:didChange:atSectionIndex:for:)
• controller(_:didChange:at:for:newIndexPath:)
• controllerDidChangeContent(_:)

I like to abstract my fetched results controllers behind a provider object. For example, an AlbumsProvider, UsersProvider, POIsProvider, and so forth. The name of the provider describes the type of object that this provider object will fetch.

Here's a simple skeleton for a UsersProvider:

class UsersProvider: NSObject {
  fileprivate let fetchedResultsController: NSFetchedResultsController<User>

  let controllerDidChangePublisher = PassthroughSubject<[Change], Never>()
  var inProgressChanges: [Change] = []

  var numberOfSections: Int {
    return fetchedResultsController.sections?.count ?? 0
  }

  init(managedObjectContext: NSManagedObjectContext) {
    let request = User.byNameRequest
    self.fetchedResultsController =
      NSFetchedResultsController(fetchRequest: request,
                                 managedObjectContext: managedObjectContext,
                                 sectionNameKeyPath: nil, cacheName: nil)

    super.init()

    fetchedResultsController.delegate = self
    try! fetchedResultsController.performFetch()
  }

  func numberOfItemsInSection(_ section: Int) -> Int {
    guard let sections = fetchedResultsController.sections,
          sections.endIndex > section else {
      return 0
    }

    return sections[section].numberOfObjects
  }

  func object(at indexPath: IndexPath) -> User {
    return fetchedResultsController.object(at: indexPath)
  }
}

I'll show you the NSFetchedResultsControllerDelegate methods that should be implemented in a moment. Let's go over this class first.

The UsersProvider class contains two properties that you wouldn't need when you're using a diffable data source:

let controllerDidChangePublisher = PassthroughSubject<[Change], Never>()
var inProgressChanges: [Change] = []

The first of these two properties provides a mechanism to tell a view controller that the fetched results controller has informed us of changes. You could use a different mechanism like a callback to achieve this, but I like to use a publisher.

The second property provides an array that's used in the NSFetchedResultsControllerDelegate to collect the different changes that our fetched result controller sends us. These changes are communicated through multiple delegate callbacks because there's one call to a delegate method for each object or section that's changed.

The rest of the code in UsersProvider is pretty straightforward. We have a computed property to extract the number of sections in the fetched results controller, a method to extract the number of items in the fetched results controller, and lastly a method to retrieve an object for a specific index path.

Note that the controllerDidChangePublisher published an array of Change objects. Let's see what this Change object looks like next:

enum Change: Hashable {
  enum SectionUpdate: Hashable {
    case inserted(Int)
    case deleted(Int)
  }

  enum ObjectUpdate: Hashable {
    case inserted(at: IndexPath)
    case deleted(from: IndexPath)
    case updated(at: IndexPath)
    case moved(from: IndexPath, to: IndexPath)
  }

  case section(SectionUpdate)
  case object(ObjectUpdate)
}

The Change enum is an enum I've defined to encapsulate changes in the fetched result controller's data.

Now let's move on to the delegate methods. I'll show them all in one go:

extension AlbumsProvider: NSFetchedResultsControllerDelegate {
  func controllerWillChangeContent(_ controller: NSFetchedResultsController<NSFetchRequestResult>) {
    inProgressChanges.removeAll()
  }

  func controller(_ controller: NSFetchedResultsController<NSFetchRequestResult>, didChange sectionInfo: NSFetchedResultsSectionInfo, atSectionIndex sectionIndex: Int, for type: NSFetchedResultsChangeType) {
    if type == .insert {
      inProgressChanges.append(.section(.inserted(sectionIndex)))
    } else if type == .delete {
      inProgressChanges.append(.section(.deleted(sectionIndex)))
    }
  }

  func controller(_ controller: NSFetchedResultsController<NSFetchRequestResult>, didChange anObject: Any, at indexPath: IndexPath?, for type: NSFetchedResultsChangeType, newIndexPath: IndexPath?) {
    // indexPath and newIndexPath are force unwrapped based on whether they should / should not be present according to the docs.
    switch type {
    case .insert:
      inProgressChanges.append(.object(.inserted(at: newIndexPath!)))
    case .delete:
      inProgressChanges.append(.object(.deleted(from: indexPath!)))
    case .move:
      inProgressChanges.append(.object(.moved(from: indexPath!, to: newIndexPath!)))
    case .update:
      inProgressChanges.append(.object(.updated(at: indexPath!)))
    default:
      break
    }
  }

  func controllerDidChangeContent(_ controller: NSFetchedResultsController<NSFetchRequestResult>) {
    controllerDidChangePublisher.send(inProgressChanges)
  }
}

There's a bunch of code here, but the idea is quite simple. First, the fetched results controller will inform us that it's about to send us a bunch of changes. This is a good moment to clear the inProgressChanges array so we can populate it with the changes that we're about to receive.

The following two methods are called by the fetched results controller to tell us about changes in objects and sections. A section can only be inserted or deleted according to the documentation.

Managed objects can be inserted, moved, deleted, or updated. Note that a moved object might also be updated (it usually is because it wouldn't have moved otherwise). When this happens, you're only informed about the move.

When the fetched results controller has informed us about all changes, we can call send on the controllerDidChangePublisher so we send all changes that were collected to subscribers of this publisher. Usually that subscriber will be your view controller.

Note:
I'm assuming that you understand the basics of Combine. Explaining how publishers work is outside of the scope of this article. If you want to learn more about Combine you can take a look at my free blog posts, or purchase my Practical Combine book.

In your view controller, you'll want to have a property that holds on to your data provider. For example, you might add the following property to your view controller:

let usersProvider: UsersProvider

Your data sources should typically be injected into your view controllers, but view controllers can also initialize their own data provider. Choose whichever approach works best for your app.

What's more interesting is how you should respond to change arrays that are sent by controllerDidChangePublisher. Let's take a look at how I subscribe to this publisher in viewDidLoad():

override func viewDidLoad() {
  super.viewDidLoad()

  // setup code...

  albumsProvider.controllerDidChangePublisher
    .sink(receiveValue: { [weak self] updates in
      var movedToIndexPaths = [IndexPath]()

      self?.collectionView.performBatchUpdates({
        for update in updates {
          switch update {
          case let .section(sectionUpdate):
            switch sectionUpdate {
            case let .inserted(index):
              self?.collectionView.insertSections([index])
            case let .deleted(index):
              self?.collectionView.deleteSections([index])
            }
          case let .object(objectUpdate):
            switch objectUpdate {
            case let .inserted(at: indexPath):
              self?.collectionView.insertItems(at: [indexPath])
            case let .deleted(from: indexPath):
              self?.collectionView.deleteItems(at: [indexPath])
            case let .updated(at: indexPath):
              self?.collectionView.reloadItems(at: [indexPath])
            case let .moved(from: source, to: target):
              self?.collectionView.moveItem(at: source, to: target)
              movedToIndexPaths.append(target)
            }
          }
        }
      }, completion: { done in
        self?.collectionView.reloadItems(at: movedToIndexPaths)
      })
    })
    .store(in: &cancellables)
}

This code is rather long but it's also quite straightforward. I use UICollectionView's performBatchUpdates(_:completion:) method to iterate over all changes that we received. I also define an array before calling performBatchUpdates(_:completion:). This array will hold on to all index paths that were the target of a move operation so we can reload those cells after updating the collection view (the app will crash if you move and reload a cell).

By checking whether a change matches the section or object case I know what kind of a change I'm dealing with. Each case has an associated value that describes the change in more detail. Based on this associated value I can insert, delete, move, or reload cells and sections.

I haven't shown you the UICollectionViewDataSource methods that are needed to provide your collection view will data and cells. I'm sure you know how to do this as it'd be no different from a very plain and boring collection view. Just make sur eto use your data provider's convenient helpers to determine the number of sections and objects in your collection view.

In Summary

Doing all this work is certainly less convenient than using a diffable data source snapshot, but in the end you'll find that when you're using a fetchBatchSize this is approach will make sure your fetched results controller doesn't make a ton of unwanted extra fetch requests.

I'm not sure whether the behavior we see with diffable data sources is expected, but it's most certainly inconvenient. Especially when you have a large set of data, fetchBatchSize should help you reduce the time it takes to load data. When your app then proceeds to fetch all data anyway except with many small requests you'll find that performance is actually worse than it was when you fetched all data in one go.

If you don't want to do any extra work and have a small data set of maybe a couple dozen items, it might be a wise choice to not use fetchBatchSize if you want to utilize diffable data source snapshots. It takes a bunch of extra work to implement fetched results controller without it, and this extra work might not be worth the trouble if you're not seeing any problems in an app that doesn't use fetchBatchSize.

I will publish a follow-up post that details a fix for the same problem in SwiftUI when you use the @FetchRequest property wrapper. If you have any feedback or questions about this post, you can reach out to me on Twitter. If you want to learn more about Core Data, fetched results controllers and analyziing performance in Core Data apps, check out my Practical Core Data book.