Skip to content

Import_(development)

William Desportes edited this page Mar 18, 2020 · 8 revisions

General Information

Import plug-ins should be placed in the libraries/plugins/import directory. Plug-ins should build upon the following code framework. Official plug-ins need to have external string definitions to keep translation efforts centralized. The definition should be added to the appropriate language file (located in the lang/ directory), if you create plug-ins for your own personal use, you can place the strings directly into this file and they will be displayed correctly.

For more detailed documentation of relevant functions and classes, please see phpMyAdmin's auto-generated phpDoc page.

Type-Detection API

The type-detection module analyzes the contents of an import and determines the appropriate MySQL data types for each column of each table.

This can be useful in several scenarios

  • The MySQL knowledge required of the user is now lowered considerably (i.e. they don't have to manually create the databases/tables/etc.)
  • You don't know the sized or types of the data and there is simply too much to shift through by hand
  • And (as an extension of the last example) it is a time saver. Even if you do know the sizes/types of each column of data you are importing you no longer have to type them all in.

Using the new type-detection module allows you to add automatic structure creation to a new import module of your creation or an existing one. API documentation follows directly below. There is example usage at the end of this page.

Documentation

The type-detection API is rather simple. The most difficult part of implementing this functionality will be converting the raw import data into a format that the type-detection module can work with. This is explained below.

There are only two functions that you will utilize:

  • $this->import->analyzeTable(&$table)

This function analyzes a table of data to determine the best-fit MySQL data types. It is used in the analysis stage.

  • $this->import->buildSQL($db_name, &$tables, &$analyses = NULL, &$additional_sql = NULL, $options = NULL, &$sql_data)

This function creates the necessary SQL statements to accomplish the import with the desired options. It is used in the execution stage.

Import Stages

There are three distinct stages that your plug-in must follow in order to implement type-detection. They are as follows:

  • Accumulation Stage

You must convert the raw import data into the type-detection format. The format can be seen in the second code example on this page.

The way you go about this depends on the file format you are working with. That is for you to determine, but generally speaking you work "upwards," by which I mean cell -> row -> table -> database, accumulating each as you progress.

  • Analysis Stage

Each table is analyzed and the best-fit types are determined. You need to store the return values of calls to this function in an dedicated array.

  • Execution Stage

The data and the analyses are used to create SQL statements which are executed.

Example Templates

Normal Import Module Template

This template shows how to create a normal import module (as opposed to one with type-detection/auto-structure-creation).

<?php
/**
 * [Name] import plugin for phpMyAdmin
 *
 * @package    PhpMyAdmin-Import
 * @subpackage [Name]
 */
declare(strict_types=1);

namespace PhpMyAdmin\Plugins\Import;

use PhpMyAdmin\Import;
use PhpMyAdmin\Plugins\ImportPlugin;

/**
 * Handles the import for the [Name] format
 *
 * @package PhpMyAdmin-Import
 */
class Import[Name] extends ImportPlugin
{
    /**
     * optional - declare variables and descriptions
     *
     * @var type
     */
    private $_myOptionalVariable;

    /**
     * Constructor
     */
    public function __construct()
    {
        parent::__construct();
        $this->setProperties();
    }

    /**
     * Sets the import plugin properties.
     * Called in the constructor.
     *
     * @return void
     */
    protected function setProperties()
    {
        $importPluginProperties = new PhpMyAdmin\Properties\Plugins\ImportPluginProperties();
        $importPluginProperties->setText('[name]');             // the name of your plug-in
        $importPluginProperties->setExtension('[ext]');         // extension this plug-in can handle
        $importPluginProperties->setOptionsText(__('Options'));

        // create the root group that will be the options field for
        // $importPluginProperties
        // this will be shown as "Format specific options"
        $importSpecificOptions = new
        PhpMyAdmin\Properties\Options\Groups\OptionsPropertyRootGroup(
            "Format Specific Options"
        );

        // general options main group
        $generalOptions = new PhpMyAdmin\Properties\Options\Groups\OptionsPropertyMainGroup(
            "general_opts"
        );

        // optional :
        // create primary items and add them to the group
        // type - one of the classes listed in libraries/properties/options/items/
        // name - form element name
        // text - description in GUI
        // size - size of text element
        // len  - maximal size of input
        // values - possible values of the item
        $leaf = new PhpMyAdmin\Properties\Options\Items\RadioPropertyItem(
            "structure_or_data"
        );
        $leaf->setValues(
            array(
                'structure' => __('structure'),
                'data' => __('data'),
                'structure_and_data' => __('structure and data')
            )
        );
        $generalOptions->addProperty($leaf);

        // add the main group to the root group
        $importSpecificOptions->addProperty($generalOptions);

        // set the options for the import plugin property item
        $importPluginProperties->setOptions($importSpecificOptions);
        $this->properties = $importPluginProperties;
    }

    /**
     * Handles the whole import logic
     *
     * @param array &$sql_data 2-element array with sql data
     *
     * @return void
     */
    public function doImport(&$sql_data = array())
    {
        // get globals (others are optional)
        global $error, $timeout_passed, $finished;

        $buffer = '';
        while (! ($finished && $i >= $len) && ! $error && ! $timeout_passed) {
            $data = $this->import->getNextChunk();
            if ($data === false) {
                // subtract data we didn't handle yet and stop processing
                $GLOBALS['offset'] -= strlen($buffer);
                break;
            } elseif ($data === true) {
                // Handle rest of buffer
            } else {
                // Append new data to buffer
                $buffer .= $data;
            }
            // PARSE $buffer here, post sql queries using:
            $this->import->runQuery($sql, $verbose_sql_with_comments, $sql_data);
        } // End of import loop
        // Commit any possible data in buffers
        $this->import->runQuery('', '', $sql_data);
    }


    // optional:
    /* ~~~~~~~~~~~~~~~~~~~~ Getters and Setters ~~~~~~~~~~~~~~~~~~~~ */


    /**
     * Getter description
     *
     * @return type
     */
    private function _getMyOptionalVariable()
    {
        return $this->_myOptionalVariable;
    }

    /**
     * Setter description
     *
     * @param type $my_optional_variable description
     *
     * @return void
     */
    private function _setMyOptionalVariable($my_optional_variable)
    {
        $this->_myOptionalVariable = $my_optional_variable;
    }
}
?>
Clone this wiki locally