Migrate module: file handling

Migrate module: file handling

Posted by stella on Tue, 2009-10-27 19:31 in

UPDATE: In the latest version of the migrate module, the hook names have changed - the word 'destination' has been removed.

So you're migrating pages and users from another CMS to Drupal using the Migrate module, but how do you handle all those file attachments? This example will cover taking the input filename and adding it to an imagefield on a Drupal content type. However, this should work in exactly the same way for filefields too.

The first thing you need to do is to take all files to be migrated and place them in your site's files directory. If you plan on using the migrate module's "clear imported records" feature, so as to do multiple test runs, you should retain a backup of these files as when the nodes are cleared / deleted, so are the files.

Create the CCK content type and add an imagefield to it to store the migrated image. Then add a content set to map the source content fields to their Drupal equivalents. When configuring the content set, you will see three fields for your image: "fid", "list" and "data". We're going to ignore these and instead set the field value using the hook_migrate_destination_prepare_node().

<?php
function mymodule_migrate_destination_prepare_node(&$node, $tblinfo, $row) {
 
$extensions = variable_get('upload_extensions_default', 'jpg jpeg gif png txt html doc xls pdf ppt pps odt ods odp');
  if (
$node->type == 'news') {
   
$filepath = 'sites/all/files/images/'. $row->cms_news_table_image;

   
// Ensure the file exists.
   
if (file_exists(file_create_path($filepath))) {
     
// Check if the file exists in the 'files' table already.
     
$file = db_fetch_object(db_query("SELECT * FROM {files} WHERE filepath = '%s'", $filepath));

     
// Create the file entry in the 'files' table if necessary.
     
if (!$file->fid) {
       
$file = new stdClass;
       
$file->filepath = $filepath;
       
$file->filename = file_munge_filename(basename($file->filepath), $extensions);
       
$file->filemime = file_get_mimetype($file->filename);
       
$file->uid = 1;
       
$file->status = 1;
       
$file->timestamp = $node->created;
       
$file->filesize = filesize($file->filepath);
       
drupal_write_record('files', $file);
      }

     
// Set up the image field.
     
$node->field_news_image[0] = array(
       
'fid' => $file->fid,
       
'list' => 1,
       
'data' => array('alt' => $node->title, 'title' => $node->title, 'description' => $node->title),
       
'uid' => $node->uid,
       
'filename' => $file->filename,
       
'filepath' => $file->filepath,
       
'filemime' => $file->filemime,
       
'status' => 1,
       
'timestamp' => $node->created,
      );

    }
   
// Print an error message if the file doesn't exist.
   
else {
     
drupal_set_message(t('Failed to find file: %file', array('%file' => $filepath)), 'error');
    }
  }
}
?>

In our example, we used a content type called "news" with a field_news_image imagefield, while the column containing the image filename in the source CMS is cms_news_table_image.

The hook_migrate_destination_prepare_node() hook is invoked just before each node record is migrated, allowing the module to manipulate the node data, one row at a time, immediately before it is stored to Drupal's database.

For each row, we check that the file we wish to migrate exists in the "files" directory and create a record for it in Drupal's files table if one doesn't already exist. Finally we configure the field_news_image field with all the required information.

Where do I put this piece of code?

I'm just starting Drupal, so I was just wondering where to put this code. Do I alter the migrate module or should it go into a separate module or do I paste it in a drupal form somewhere?

Posted by Roadie (not verified) on Fri, 2009-11-20 20:55
Custom module

It's probably best to put it in a custom module for your site. Don't modify the migrate module itself.

Posted by stella on Sat, 2009-11-21 01:25
More generic solution

I wonder if it would be possible to make a 'node_migrate' module, that works on migrating Drupal to Drupal, in a as generic way as possible?

Anyone know of a module to do that?

Posted by a_c_m (not verified) on Wed, 2010-03-17 14:17
migrate extras module

Well since I wrote this post, the Migrate Extras module was released. It doesn't handle file attachments IIRC, but does handle file/image fields. It's really where things should go.

I also think that there are plans to create different sets of mappings, eg. Joomla 1.5 to Drupal 6 content set mappings plus code, and a different package for CMS Made Simple to Drupal in another.

Posted by stella on Thu, 2010-03-18 12:05
Ahh ok, well i'm going to

Ahh ok, well i'm going to make a start on D5 -> D6 anyway. Help from anyone very welcome http://etherpad.com/c8hZ6InzJk

Posted by a_c_m (not verified) on Thu, 2010-03-18 15:26
Thank you! Great snippet for

Thank you! Great snippet for my custom migration!

Posted by Miche (not verified) on Mon, 2011-02-14 19:04