[PROPOSAL] Import items without saving directly

[WouterFlorijn]

Prerequisites

Versions

• PHP version: 7.1.9
• Laravel version: 5.7.19
• Package version: 3.0+

Description

Importing models should be possible without saving them directly. It should also be possible to return anything other than models from an import.

I currently can not upgrade from 2.1.30 to 3.0.0 or higher. I need to be able to import a set of models without saving them. I use this for tests. Also, I have a test where a row contains information to create two related models (of different classes). Currently, I can only have one model per row. Restructuring the CSV file isn't possible here.

IMO, 3.0.0+ is too tightly focused on the ideal situation where one CSV file corresponds to one table, and one row corresponds to one model, and every model should be saved instantly. This wasn't the case for 2.X, and I don't know why it was changed.

Note that I don't want to stay on 2.1.30, because it requires the abandoned phpoffice/phpexcel package.

Example

An additional method should be added to the Excel facade, which has the same signature as import, but returns models instead of saving them. Example:

$users = Excel::load(new UsersImport, 'users.xlsx');

Additionally, an item method should be added to import classes, that can return any type instead of only models.

[patrickbrouwers]

Using ToModel is just one way of dealing with imports. This is just option that makes 1 model per row inserts easier.

If you want to handle different structures and in a way that it's similar to version 2.1, than you need to use ToCollection: https://laravel-excel.maatwebsite.nl/3.1/imports/collection.html In the collection method you get a similar collection of rows that you got in 2.1.

If you want get the models only in the controller and not in your import object (not really a recommended approach) than there's the ::toCollection method, which returns the raw row (no conversion to models is done) to your controller.
(https://laravel-excel.maatwebsite.nl/3.1/imports/basics.html#importing-to-array-or-collection)

$users = Excel::toCollection(new UsersImport, 'users.xlsx');

[WouterFlorijn]

@patrickbrouwers thanks for your reply. So if I understand correctly, using Excel::import on an import that implements ToCollection will return a collection of models without saving them? If so, this is not clearly stated in the documentation, as the example at the bottom of https://laravel-excel.maatwebsite.nl/3.1/imports/collection.html is as follows:

public function import() 
{
    Excel::import(new UsersImport, 'users.xlsx');
}

The return value isn't stored anywhere, implying that the imported collection of models is saved directly (or at least that the method has side-effects). Otherwise it would be useless calling import without assigning the return value to a variable right?

[patrickbrouwers]

@WouterFlorijn No, ::import() will never return any rows or models. Only Excel::toCollection() does.

Using the ToCollection concern is something different than using Excel::toCollection().
When implementing ToCollection the entire import is encapsulated within the Import object. Returning something in ToCollection won't be returned to the controller.

[WouterFlorijn]

@patrickbrouwers Just to get back, I still think we're misunderstanding each other. What I'm simply looking for is a method that returns a Collection of Models, without saving them. I don't want to process the rows in my controller, I would like to do that in an Import class. So far I haven't found this.

The idea:

TransactionsImport.php:

<?php

namespace App\Imports;

use App\Banks\Transaction;
use Carbon\Carbon;
use Maatwebsite\Excel\Concerns\ToModel;
use Maatwebsite\Excel\Concerns\WithHeadingRow;

class TransactionsImport implements ToModel, WithHeadingRow
{
    public function model(array $row)
    {
        $transaction = new Transaction;
        $transaction->amount = $row['amount'];
        $transaction->currency = $row['currency'];

        return $transaction;
    }
}

TransactionsController.php:

...
$transactions = Excel::someFunction(new TransactionsImport, 'path/to/file.csv');
// $transactions is a Collection of Transaction models that are not saved to the database yet.
...

I also found two issues with the toCollection function:

1. It returns a Collection with a single item, which is another Collection that contains a Collection for each row.
2. It takes an import argument, which is useless in cases where you just want to return the rows as Collections.

[GlennM]

What I'm simply looking for is a method that returns a Collection of Models, without saving them. I don't want to process the rows in my controller, I would like to do that in an Import class. So far I haven't found this.

Perhaps these docs are helpful to you:
Handling persistence on your own
Architecture Concepts

[thrazu]

@WouterFlorijn you can try this approach to get an array from your file:

$array = Excel::toArray([], 'file.xlsx');

[harshrajk]

@WouterFlorijn you can try this approach to get an array from your file:

$array = Excel::toArray([], 'file.xlsx');

That's exactly what I was looking for.

[niektenhoopen]

+1 for this request. I don't think the model() and collection() methods should save anything. It should return the data from an Excel/CSV file into a Collection (or Array) of Models/Arrays. Persistence should be handled separately, or at least be a choice.

My use case: I want to get the min and max date from records in a CSV. If this overlaps with an existing imported CSV, it should not import anything before the user confirms that the overlap is correct.

Edit: Ah, I am responding to a closed issue. How did you handle this @WouterFlorijn?

[patrickbrouwers]

You can easily achieve this yourself already by keeping state inside the import class, you could wrap up the toModels method inside a trait and re-use it in all your exports.

new UsersImport implements ToModel, WithHeadingRow
{
    use Importable;

    protected array $models;

    public function model(array $row)
    {
         $this->models[] = new User($row);
    }

    public toModels(string $filename): array
    {
       $this->import($filename);

        return $this->models;
    }
}

$nonPersistedUsers = (new UsersImport)->toModels('users.xlsx');

[giusecapo]

This would be a great feature! Specially, if you are, like me, importing an xls and you want to preview data before you import it. Any updates on this feature?

[giusecapo]

You can easily achieve this yourself already by keeping state inside the import class, you could wrap up the toModels method inside a trait and re-use it in all your exports.

new UsersImport implements ToModel, WithHeadingRow
{
    use Importable;

    protected array $models;

    public function model(array $row)
    {
         $this->models[] = new User($row);
    }

    public toModels(string $filename): array
    {
       $this->import($filename);

        return $this->models;
    }
}

$nonPersistedUsers = (new UsersImport)->toModels('users.xlsx');

Yeah, could be a workaround, but you should create two different import classes, like "UsersNotPersistedImport" and "UserPerstitedImport" to be able to create two different "model()" function, unless you can, somehow, set a "state" of your importer (like, in the constructor). Something like "needToStore": boolean; and check up on it when it's time to store, in the "model" function.

[younus93]

@WouterFlorijn you can try this approach to get an array from your file:

$array = Excel::toArray([], 'file.xlsx');

This is a perfect work-around. We have to build a small wrapper to convert these array items into model instances.

Did we find out a way to use the existing package to generate model instances and return without saving to the database?