Light-weight Core Data Migration in a Sandboxed App

This post describes the steps you need to do in order to perform a light-weight Core Data migration in a sandboxed Mac OS X application. As far as I can determine, without the steps described below, Core Data light-weight migration in a sandboxed app will fail every time.

Note that the points described in this post are strictly limited to a Sandboxed application that is shipped via the Mac App Store. If your app is not sandboxed, migration works perfectly without any additional steps.

Core Data Migration Process

Typically, you request Core Data to perform migration by using code something like the following in your NSPersistentDocument subclass:

-(BOOL)configurePersistentStoreCoordinatorForURL:(NSURL *)url ofType:(NSString *)fileType modelConfiguration:(NSString *)configuration storeOptions:(NSDictionary *)storeOptions error:(NSError **)error {
  NSMutableDictionary *options = (storeOptions != nil) ? [storeOptions mutableCopy] : [[NSMutableDictionary alloc] init];
 
  options[NSMigratePersistentStoresAutomaticallyOption] = @(YES);
  options[NSInferMappingModelAutomaticallyOption] = @(YES);
 
  return [super configurePersistentStoreCoordinatorForURL:url ofType:fileType modelConfiguration:configuration storeOptions:options error:error];
}

This tells Core Data that, where possible, you’d like it to automatically migrate your data models in the case where the user has opened a document with an old data model.

However, the problem comes when Core Data attempts to perform the migration. You see, to perform the migration, Core Data creates a temporary file to migrate the database into, and then once done, it overwrites the original file with the migrated file.

For whatever reason, the Core Data engineers chose to create the temporary file in the same directory as the original file. In a sandboxed app, however, it is almost certain that the application doesn’t have authority to create this file. As a result, the migration just aborts and you get an error that looks like:

encountered the following error: {
    NSLocalizedDescription = "The document \U201cold_model.logdive\U201d could not be opened. An error occured during persistent store migration.";
    NSUnderlyingError = "Error Domain=NSCocoaErrorDomain Code=134110 \"An error occured during persistent store migration.\" 
    UserInfo=0x1002d8b30 {sourceURL=file://localhost/Users/edwardsc/source/tmp/old_model.logdive, 
    reason=Can't copy source store to destination store path, destinationURL=file://localhost/Users/edwardsc/source/tmp/.old_model.logdive.migrationdestination_41b5a6b5c6e848c462a8480cd24caef3, 
    NSUnderlyingError=0x1002daec0 \"The operation couldn\U2019t be completed. (NSSQLiteErrorDomain error 14.)\"}";
    destinationURL = "file://localhost/Users/edwardsc/source/tmp/.old_model.logdive.migrationdestination_41b5a6b5c6e848c462a8480cd24caef3";
    reason = "Can't copy source store to destination store path";
    sourceURL = "file://localhost/Users/edwardsc/source/tmp/old_model.logdive";
}

The Hacky Work-Around

In an earlier post, I described a way that you could workaround this Core Data limitation by creating temporary entitlements. However, it turns out that our friends at Apple will reject your app submission if you use this technique (which is probably fair enough).

The Real Fix

So, what is the Apple-recommended way to resolve this problem? I opened a DTS to get some feedback from Apple, and it basically boils down to performing the following steps:

Override the makeDocumentWithContentsOfURL:ofType:error: method.
Let it try to auto-migrate the document (this may work if the document happens to be in a Core Data-writable directory)
Upon migration failure, copy the file to a writable directory
Ask Core Data to migrate the temporary file
Copy the migrated file back to the original location
Allow the document opening process to continue

So, let’s get into the specific steps that are required to make this happen.

Subclassing NSDocumentController

The first thing that you will need to do is create a class that subclasses NSDocumentController (if you don’t already have one).

Next, you need to make Core Data pick up your instance instead of the standard NSDocumentController. The process for registering your custom class is slightly unusual in that chooses the first document controller that has been instantiated, so what you need to do is instantiate an instance of your class very early on in your application lifecycle (note that you don’t need to save a reference to it; just make sure it has been created).

The way that was recommended to me by the Apple DTS engineer was to add an Object in my MainMenu.xib and set its class name to my subclass name. This will force an instance to be instantiated when the main menu nib gets loaded.

Implementing Migration Steps

According to my Apple DTS engineer, the correct method to hook into is makeDocumentWithContentsOfURL:ofType:error:, so in your newly created subclass of NSDocumentController add the following code:

/**
 * Overrides the default NSDocumentController method of the same name to 
 * handle lightweight migration when running in a sandboxed environment.
 *
 * The basic logic is as follows:
 *   * Attempt to open the document as normal.  If the document is able to 
 *     be opened, then we can just return as normal.  If the document happens 
 *     to be in a directory that the sandbox can write to, lightweight 
 *     migration should work OK and processing will just finish at this point.
 *   * If the document couldn't be opened, we check to see if it was a 
 *     migration error.  If it was then we'll have a crack at migrating in a 
 *     temporary directory.
 *   * Copy the original file into a temporary file, and then ask CoreData to 
 *     open that file as if it were the one selected by the user.  At this 
 *     point, CoreData will try to perform lightweight migration on the 
 *     temporary file.
 *   * If that wen't well, we attempt to replace the original file with the 
 *     freshly migrated temporary file.
 *   * Assuming that the move back was successful, we now hand off to CoreData 
 *     to open the original (or at least, the replaced original) file.
 *
 * Any errors during the copying/migrating/moving back will just bail out of 
 * the whole process.
 *
 * Note that this technique still relies on your document requesting for the 
 * lightweight migration to occur by setting the NSMigratePersistentStoresAutomaticallyOption 
 * and NSInferMappingModelAutomaticallyOption options to YES in your document 
 * subclass' implementation of:
 *
 *     [NSPersistentDocument configurePersistentStoreCoordinatorForURL:ofType:modelConfiguration:storeOptions:error:]
 */
-(id)makeDocumentWithContentsOfURL:(NSURL *)url ofType:(NSString *)typeName error:(NSError **)error {
  LDDocument *document = [super makeDocumentWithContentsOfURL:url ofType:typeName error:error];
  if (document == nil) {
    // couldn't open the file.  Let's see if it was a migration error...
    NSInteger errorCode = [*error code];
    if (errorCode == NSMigrationError) {
      // determine a temporary file name and make sure it doesn't exist
      NSURL *tmpFileURL = [self tmpFileURL];
      [[NSFileManager defaultManager] removeItemAtURL:tmpFileURL error:error];
 
      // now, let's copy the original store into the tmporary file
      if ([[NSFileManager defaultManager] copyItemAtURL:url toURL:tmpFileURL error:error]) {
        // and give CoreData a crack at migrating the temporary one
        NSPersistentDocument *migratedDocument = [super makeDocumentWithContentsOfURL:tmpFileURL ofType:typeName error:error];
        if (migratedDocument != nil) {
          // excellent!  CoreData was able to successfully migrate the temporary file, so now we need
          // to move the migrated file back on top of the original file.
          if ([[NSFileManager defaultManager] replaceItemAtURL:url withItemAtURL:tmpFileURL backupItemName:nil options:0 resultingItemURL:&url error:error]) {
            // go through the normal document opening process
            return [super makeDocumentWithContentsOfURL:url ofType:typeName error:error];
          }
        }
      }
    }
  }
  return document;
}
 
-(NSURL *)tmpFileURL {
  NSString *filename = [NSString stringWithFormat:@"%@-migration.logdive", [[NSUUID UUID] UUIDString]];
  return [NSURL fileURLWithPath:[NSTemporaryDirectory() stringByAppendingPathComponent:filename]];
}

And that is it! Now everything works just fine.

I’ve only just resubmitted my app to the store, but I am not expecting a rejection this time as I no longer have the dodgy temporary entitlements.

Closing Thoughts

I have a couple of general comments about the above approach:

If it is a very large document, it may take a non-trivial amount of time to copy to the temporary location, perform the migration, and then copy back. I initially considered moving the original file instead of copying (which would mitigate some of this concern), however, I decided on a copy operation because I didn’t want to leave the user in a situation where I have moved their original file, something goes wrong, and then all of a sudden, their file is no longer where it used to be. That would be ugly indeed.
I have no way to tell (with 100% certainty) that once the migration has completed whether Core Data has flushed all the migrated data out to the temporary store. My DTS engineer said that it should be OK, and my testing indicates that it is true, but a tiny bit of nervousness remains.
I do believe that this whole process is a defect with the Core Data light-weight migration process. As far as I can tell, they have provided a migration process that will fail virtually 100% of the time. I plan on submitting a Radar suggesting that they create their temporary migration file in a writable directory instead of the current directory. That would solve this problem and remove the need for developers all over the world to have code like the above.