Migrate module: migrating a node's taxonomy terms

Migrate module: migrating a node's taxonomy terms

Posted by stella on Wed, 2010-03-10 10:42 in

One of the issues I encountered when migrating nodes to Drupal, using the migrate module, was that I couldn't associate nodes with more than one taxonomy term. Actually in this example, I'm migrating content from one Drupal database to another, so I'm going to assume everyone is already familiar with the database structure, specifically the node and term_node tables.

When I first started using the migrate module, I ran into a similar problem with migrating a user's roles. It's not possible to just create a Views relationship (aka LEFT JOIN) between the node and term_node tables using the node id. This will produce one row for each node and taxonomy combination, but the migrate module is only able to handle data sets that contain one row for each entity. With the above solution, I have more than one row for each node, which causes the migrate module to import the same node more than once, causing all sorts of problems.

Like with the user roles example before, we can overcome this by implementing a migrate hook, specifically hook_migrate_prepare_node().

<?php
/**
* Implements hook_migrate_prepare_node().
*/
function mymodule_migrate_prepare_node(&$node, $tblinfo, &$row) {
  static
$vocabs, $source_vocabs;
 
$errors = array();

 
// Get a list of vocabs in our target database.
 
if (empty($vocabs)) {
   
$result = db_query("SELECT vid, name FROM {vocabulary}");
    while (
$vrow = db_fetch_object($result)) {
     
$vocabs[$vrow->name] = $vrow->vid;
    }
  }

 
// Set up per-node type specific stuff.
 
$node_vocabs = array();
  switch (
$node->type) {
    case
'event':
     
// Here the 1 and 0 identify which are free-tagging vocabs and which aren't.
     
$node_vocabs = array('Regions' => 0, 'Keywords' => 1, 'Topics' => 0);
      break;
     case
'news':
     
$node_vocabs = array('Keywords' => 1, 'Topics' => 0);
      break;
  }

 
// I have 2 database connections defined in my settings.php.
  // This statement allows me to use the source Drupal database for subsequent queries.
 
db_set_active('old_drupal_db');

 
// Get vocabs from our source database.
 
if (empty($source_vocabs)) {
   
$result = db_query("SELECT vid, name FROM {vocabulary}");
    while (
$vrow = db_fetch_object($result)) {
     
$source_vocabs[$vrow->name] = $vrow->vid;
    }
  }


 
// Map each node to its taxonomy terms.
 
if (!empty($node_vocabs)) {
    foreach (
$node_vocabs as $vname => $tags) {
     
$terms = array();
     
$result = db_query("SELECT d.name FROM {term_data} d, {term_node} n WHERE n.tid = d.tid AND n.nid = %d AND d.vid = %d", $row->nid, $source_vocabs[$vname]);
      while (
$term = db_fetch_object($result)) {
       
$terms[] = $term->name;
      }
     
     
// Depending on whether it's a free-tagging vocabulary or not, the terms are stored slightly differently.
     
$vid = $vocabs["$vname"];
     
$vid_key = 'migrate_taxonomy_' . $vid;
      if (!empty(
$terms)) {
        if (
$tags) {
         
$node->$vid_key = '"' . implode('"' . $tblinfo->multiple_separator . '"', $terms) . '"';
        }
        else {
         
$node->$vid_key = implode($tblinfo->multiple_separator, $terms);
        }
      }
    }
  }

 
// Switch back to using the default, aka target, Drupal database.
 
db_set_active('default');

  return
$errors;
}
?>

Note, for each vocab for a node type, I identify whether or not it's a free-tagging one or not. This is because I handle them slightly differently because of the way taxonomy module treats them, and to avoid problems with commas and quotes within terms, etc. In addition, there can be problems if you use commas as your separator, so when creating the content set I set the multiple separator ($tblinfo->multiple_separator) to be a pipe |.

thanks, and another approach

Thanks for these writeups, stella -- Migrate module is awesome but using it for a serious project does mean digging in to details like this. I've been learning it and Table Wizard it the past few weeks, to convert an expressionengine site into drupal. What a joy! It's great to be able to focus on the rules and avoid any sort of tedious manual work.

For converting taxonomy terms I went with a different approach. I leave the terms out of the initial query entirely, and pull them in during preprocessing. First I have a view that takes an old id as an argument and returns a simple array of term ids from the old system. Then I convert those ids into the corresponding drupal tids using a straightforward conversion function. Finally I use these tids to add terms to the node.

Note that this code has a number of assumptions that are fine for my current task but which might be a problem for others.

<?php
/**
* Implementation of hook_migrate_prepare_node().
*/
function example_migrate_prepare_node(&$node, $tblinfo, $row) {
 
$errors = array();
  switch (
$node->type) {
    case
'client':
     
// Convert taxonomy terms. Assumes no free-tagging.
     
$old_cat_ids = _get_old_tids($node->field_client_id[0]['value']);
      if(
$old_cat_ids) {
       
$tids = _get_new_tids($old_cat_ids);
        foreach(
$tids as $tid) {
         
$node->taxonomy[] = taxonomy_get_term($tid);
        }
      }
     
// Other preprocessing goes here
     
break;
  }
  return
$errors;
}

/*
* Fetch the old category IDs for an old entry.
* Depends on a view 'exp_terms' that returns terms given an id as an argument.
*
* @param $oldid
*  Entry ID in the old system
* @return
*  Array of old-system term IDs
*/
function _get_old_tids($oldid) {
 
$result = views_get_view_result('exp_terms', 'default', array($oldid));
  if (
$result) {
   
$tids = array();
    foreach(
$result as $term) {
     
$tids[] = $term->exp_category_posts_cat_id;
    }
    return
$tids;
  }
  return
NULL;
}

/*
* Convert old category term ids to new tids.
* The "lookup table" is hard-coded here but could be dynamic
* if Migrate module was used to import the terms.
*
* @param $oldids
*  Array of old category terms
* @return
*  Array of Drupal-side tids
*/
function _get_new_tids($oldids) {
 
$cat_id = array(); // drupal tid=>old ee cat_id
 
  // client categories
 
$cat_id['1'] = '47'; // education
 
$cat_id['2'] = '40'; // health
 
$cat_id['3'] = '14'; // print ident
 
$cat_id['4'] = '15'; // interactive
 
 
$result = array();
  foreach(
$oldids as $id) {
   
$newid = array_search($id, $cat_id);
    if (
$newid) {
     
$result[] = $newid;
    }
  }
  return
$result;
}
?>

Posted by markabur (not verified) on Wed, 2010-03-10 22:43
Thanks for the detail Stella,

Thanks for the detail Stella, thought I'd share my method of doing it. I've got the full details in a blog post:

http://openmonkey.blogspot.com/2011/09/drupal-migrate-multiple-taxonomy-terms.html

The basic gist is that you can use some built in migrate functions in the field mapping in combination with your db's string concat functionality to do this in just a few steps.

Posted by Reuben (not verified) on Wed, 2011-09-21 14:08