Writing custom JSON encoding and decoding logic

The default behavior for Codable is often good enough, especially when you combine this with custom CodingKeys, it's possible to encode and decode a wide variety of JSON data without any extra work.

Unfortunately, there are a lot of situations where you'll need to have even more control. The reasons for needing this control are varied. You might want to flatten a deeply nested JSON structure into a single Codable object. Or maybe you want to assign a default value to a property if it's not possible to extract this value from the received JSON data. Or maybe you want to make your an NSManagedObject subclass work with Codable.

No matter what your reason for needing to implement custom JSON encoding or decoding logic is for your Codable objects, the approach is always the same. In this post you'll learn how you can implement a custom init(from:) to decode JSON data, and a custom encoding(using:) method to encode a Codable object to JSON data.

Implementing custom JSON decoding logic

Decoding JSON data into a Decodable object is done through a special initializer that's required by the Decodable protocol. The initializer is called init(from decoder: Decoder), or as I like to write it init(from:).

This initializer is normally generated for you, but you can also implement it yourself if you need an extremely high level of customization.

The init(from:) initializer receives an object that conforms to the Decoder protocol, and it could be a JSONDecoder but that's not guaranteed. Swift's Decodable protocol was designed so it could work with different kinds of data.

In your initializer, you'll obtain a container object that knows how to extract values from the Data that's being decoded using your CodingKeys to look up these values. Note that as soon as you define your own init(from:), Swift will no longer generate your CodingKeys enum for you (even though Swift will generate an init(from:) if you define your own CodingKeys).

Let's look at a simple example of a custom init(from:):

struct User: Decodable {
  enum CodingKeys: String, CodingKey {
    case id, fullName, isRegistered, email
  }

  let id: Int
  let fullName: String
  let isRegistered: Bool
  let email: String

  init(from decoder: Decoder) throws {
    let container = try decoder.container(keyedBy: CodingKeys.self)
    self.id = try container.decode(Int.self, forKey: .id)
    self.fullName = try container.decode(String.self, forKey: .fullName)
    self.isRegistered = try container.decode(Bool.self, forKey: .isRegistered)
    self.email = try container.decode(String.self, forKey: .email)
  }
}

This User struct is fairly standard, and if you look at it there's nothing fancy happening here. My CodingKeys all use their generated raw value which means that I expect my JSON to perfectly mirror the properties in this struct. In the init(from:) initializer, I obtain an instance of KeyedCodingContainer by calling container(keyedBy:) on the Decoder object that was passed to my initializer. This will create an instance of KeyedCodingContainer that will use the raw values for the cases on CodingKeys to look up information in my JSON data.

I can then call decode(_:forKey:) on my container object to extract an object of a given Decodable type (for example Int, String, or Bool) from my (JSON) data using the key that I passed as the forKey argument.

So for example, self.id = try container.decode(Int.self, forKey: .id) will attempt to look up a value for the key "id", and try to cast it to an Int. This is repeated for all properties on my struct.

This initial example shows how you can decode data that's consistent and always follows the same format. But what happens if the data is slightly less consistent, and we might need to work with default values in case a certain key is missing from the source data.

Assigning fallback values using a custom init(from:) method

Imagine that you are given the following JSON:

[
  {
    "id": 10,
    "fullName": "Donny Wals",
    "isRegistered": true,
    "email": "[email protected]",
  },
  {
    "id": 11,
    "fullName": "Donny Wals",
    "email": "[email protected]",
  }
]

Note how this is an array of two objects. One has an isRegistered property, and the other doesn't. You could say well, that should be a Bool? then so if isRegistered is missing, its value will be nil. That would work just fine.

However, we have a special requirement. If isRegistered is missing from the JSON data, we want the value for this property to be false.

You could attempt to define your User struct like this:

struct User: Decodable { 
  let id: Int
  let fullName: String
  let isRegistered = false
  let email: String
}

Unfortunately, this produces the following warning:

Immutable property will not be decoded because it is declared with an initial value which cannot be overwritten

That's a shame because we do want to use the isRegistered value from the JSON data if it's present. Luckily, we can achieve this through a custom init(from:) implementation.

struct User: Decodable {
  enum CodingKeys: String, CodingKey {
    case id, fullName, isRegistered, email
  }

  let id: Int
  let fullName: String
  let isRegistered: Bool
  let email: String

  init(from decoder: Decoder) throws {
    let container = try decoder.container(keyedBy: CodingKeys.self)
    self.id = try container.decode(Int.self, forKey: .id)
    self.fullName = try container.decode(String.self, forKey: .fullName)
    self.isRegistered = try container.decodeIfPresent(Bool.self, forKey: .isRegistered) ?? false
    self.email = try container.decode(String.self, forKey: .email)
  }
}

Note that all my properties are defined as non-optional. That's because we know that isRegistered should always have a value, even if we didn't receive one in our JSON data.

In the init(from:) implementation, I use decodeIfPresent instead of decode to extract the value for isRegistered. I can't do this with decode because decode assumes that a value exists for the key that you pass, and it assumes that this value has the type that you're looking for. If no value exists for the given key, or this value can't be casted to the desired type, decode will throw a decoding error.

To work around this you could use try? and hide the error, but then you might be hiding far more important mistakes. A better solution to only decode a value if it exists is to use decodeIfPresent. This method will attempt to look up a value for the given key, and if no value was found this method will return nil. If a value was found but it can't be cast to the desired type, an error is thrown because that means your Data isn't structured as expected.

Because decodeIfPresent returns and optional value (in this case Bool? because I attempt to decode to Bool), you can use a nil-coalescing operator to assign a default value (?? false).

This example is relatively simple, but it's also quite powerful. It shows how you can leverage the convenient APIs that were designed for Decoder to decode Data into a Swift object without actually knowing which type of Data you're dealing with. It's easy to forget but none of the code in init(from:) is aware that we're decoding JSON data with a JSONDecoder.

Using a custom init(from:) implementation to future proof decoding for enums

You already know that enums in Swift can be Decodable (and Encodable) as long as their raw value matches the value used in your JSON data. However, you might run into trouble and decoding failures when your service returns an enum case that you didn't know about when you defined your model. Luckily, you can use a custom init(from:) to safely decode unkown enum cases into an other case with an associated value when you encounter an unkown value.

For example, imagine a Product struct that has a status property. This status is initially defined as follows:

enum Status: String, Decodable {
  case completed, inProgress
}

This is simple enough, and will work perfect as long as your back-end only returns "completed" or "inProgress" for the value of status on a product object.

However, in programming, we often have to account for the unkown. Especially if you do not control the server, or if your back-end is maintained by a different team, you might want to make sure your status can handle other values too. After all, you might not want your decoding to fail just because you encountered an unknown status string. Especially if your server team can't provide any guarantees about whether they might add new enum cases on the server side that you don't know about.

Imagine that you need to decode the following (partial) JSON data:

[
  {
    "status": "completed"
  },
  {
    "status": "inProgress"
  },
  {
    "status": "archived"
  }
]

There's a new, unkown "archived" status in this JSON data. Normally, decoding this data would fail because your Status enum is not aware of this new value. However, a custom init(from:) will help you decode this value into a new other(String) case:

enum Status: Decodable {
  case completed, inProgress
  case other(String)

  init(from decoder: Decoder) throws {
    let container = try decoder.singleValueContainer()
    let value = try container.decode(String.self)

    switch value {
    case "completed": self = .completed
    case "inProgress": self = .inProgress
    default: self = .other(value)
    }
  }
}

Note that I've removed the String raw value for Status. That's because an enum with a raw value can't have enum cases with associated values.

In the custom init(from:), I use decoder.singleValueContainer() to obtain a container that will only decode a single value. In this case, that's the string that's uses as the value for my product's status property. I'll show you how I've defined Product in a moment to clarify this.

I ask the container to decode its single value into a String, and then I use a switch to check the value of this string, and I use it to assign the appropriate enum case to self. If I find an unkown value, I assign the decoded value as the associated type for other. This will ensure decoding always works, even if the back-end team adds a new value without hiding any errors. Instead, you can check for the other case in your code, and handle this case in a way that is appropriate for your app.

Here's what the Product struct and my decoding code looks like:

struct Product: Decodable {
  let status: Status
}

let decoder = JSONDecoder()
let products = try decoder.decode([Product].self, from: jsonData)

As you can see this all looks very standard. The main takeaway here is that you can use a single value container to extract the value of a property in your JSON that isn't a JSON object/dictionary. You'll mostly find yourself use decoder.singleValueContainer() in the context of decoding enums.

Using RawRepresentable as an alternative to enums when handling unknown values

As with many things in programming, there's more than one way to implement a future-proof Decodable model. In the previous section I showed you how to use an enum with an other case to allow the decoding of new, and unknown values. If you don't like this, you can use a slightly different approach that was pointed out to me by Ian keen.

This alternative approach involes using a struct that's RawRepresentable by a String, as well as Decodable with static values for your known values. Here's what that would look like:

struct Status: Decodable, RawRepresentable {
  typealias RawValue = String

  static let completed = Status(rawValue: "completed")
  static let inProgress = Status(rawValue: "inProgress")

  let rawValue: String

  init?(rawValue: String) {
    self.rawValue = rawValue
  }
}

The nice thing about this is that you don't need to write any custom decoding (or encoding) logic at all. If you make Status conform to Equatable, you could even write comparison logic that looks a lot like you're used to with enums:

if let product = products.first, product.status == .completed {
  print(product status is completed)
}

Personally, I don't have a strong preference for either approach in this case. If you need to handle cases where you got an unknown value explicitly, an other enum might be a little nicer since you could easily compare to other. If you can handle any value just fine and only want to make sure you can decode an unknown value, the RawRepresentable struct might be a little nicer. It's up to you to decide the better fit.

There are many more neat little tricks that you can do with custom decoders, but for now you know everything you need to know write custom decoders for the most common situations you might encounter.

Implementing custom JSON encoding logic

Now that you know about decoding data into a Decodable object, it only makes sense to take a look at encoding an Encodable object into data too. Note how I didn't say JSON data. I did that on purpose because both your custom init(from:) and encode(to:) work without knowing what the format of the data is.

The custom decoding logic that I've shown you earlier used a container object to extract and convert data into the required data types like String, Int, or even your own Decodable objects.

Let's see how we apply this knowledge to a custom encode(to:) method for the User struct that I've shown you in the section on decoding. Here's the User struct from the previous section with the encode(to:) method already added to it:

struct User: Codable {
  enum CodingKeys: String, CodingKey {
    case id, fullName, isRegistered, email
  }

  let id: Int
  let fullName: String
  let isRegistered: Bool
  let email: String

  init(from decoder: Decoder) throws {
    let container = try decoder.container(keyedBy: CodingKeys.self)
    self.id = try container.decode(Int.self, forKey: .id)
    self.fullName = try container.decode(String.self, forKey: .fullName)
    self.isRegistered = try container.decodeIfPresent(Bool.self, forKey: .isRegistered) ?? false
    self.email = try container.decode(String.self, forKey: .email)
  }

  func encode(to encoder: Encoder) throws {
    var container = encoder.container(keyedBy: CodingKeys.self)
    try container.encode(id, forKey: .id)
    try container.encode(fullName, forKey: .fullName)
    try container.encode(isRegistered, forKey: .isRegistered)
    try container.encode(email, forKey: .email)
  }
}

The code in encode(to:) is very similar to the code in init(from:). The main difference is in how you obtain a container. When you encode a struct to an Encoder, you need to obtain a mutable container that uses your CodingKeys as its mapping from your Encodable to the output format (usually JSON). In the case of encoder.container(keyedBy: CodingKeys.self), obtaining a container can't fail so you don't prefix this call with try.

Because you'll be encoding values into the container, the container needs to be a var. You'll mutate the container every time you ask it to encode a value.

To encode values, you call encode(_:forKey:) with the property you want to encode, and what key this property should be decoded to.

Sometimes, you'll want to send your encoded data to a server, and this server might expect you to omit nil values from your output. If that's the case, you should use encodeIdPresent(_:forKey:). This method will check whether the provided value is nil, and if it is, the key/value pair will be omitted from the container's output.

Next, let's take a look at how the encode(to:) for the Status enum from the previous section should be written since the Swift compiler can't properly account for the other enum case.

enum Status: Codable {
  case completed, inProgress
  case other(String)

  init(from decoder: Decoder) throws {
    let container = try decoder.singleValueContainer()
    let value = try container.decode(String.self)

    switch value {
    case "completed": self = .completed
    case "inProgress": self = .inProgress
    default: self = .other(value)
    }
  }

  func encode(to encoder: Encoder) throws {
    var container = encoder.singleValueContainer()

    switch self {
    case .completed: try container.encode("completed")
    case .inProgress: try container.encode("inProgress")
    case .other(let val): try container.encode(val)
    }
  }
}

As you might have expected, the implementation for encode(to:) looks similar to init(from:). The encode(to:) method obtains a single value container, and I use a switch to check the value of self so I can determine which string should be encoded by the single value container. If I encounter my .other case, I extract the associated value and tell my container to encode that associated value. This will make sure that we always properly encode and send our enum to the server (or that we can persist it to disc) without discarding the original unkown value.

Of course if you don't use an enum but instead opt to use the RawRepresentable alternative from the previous section, the encoding will work fine out of the box, just like the decoding did.

In Summary

In this post, you learned how you can override Swift's generated init(from:) and encode(to:) methods that are added for Decodable and Encodable objects respectively.

You learned that Swift uses a container object to read and write values from and to a JSON object, and you saw how you can use this container to extract and encode data. You also learned how you can leverage custom encoding and decoding logic to write enums that can decode cases that weren't known at the time of defining your enum. This is incredibly useful to make sure your code is as future proof as possible.

I also showed you an alternative to using an enum that's based on using a RawRepresentable struct that has static members for what would normally be your known enum cases. One of the benefits of this approach is that the Decodable and Encodable conformances can be generated by the compiler so you don't need to do any extra work.

In this post, you'll learn how you can use custom encoding and decoding logic to work with arbitrary enum cases that have associated values by passing around your Decoder object to decode the enum case and associated value seperately from the same underlying data object. This sounds similar to this Swift evolution proposal, but as you'll find out in the post it's quite different.

Another interesting thing you can do with a custom init(from:) is flattening nested data into a single struct, or expand a single struct into nested data using encode(to:). You'll learn how to do this in this post.

With the knowledge from this post you'll be able to implement highly customized JSON encoding and decoding flow up to the point where your JSON data and Codable object are almost nothing alike.