Import Module Documentation

Overview

The Import Module is the data migration engine. It enables non-technical users (Admins) to upload CSV files to bulk-create records. It is critical for onboarding (migrating from legacy systems) and daily operations (bulk stock uploads). It minimizes data entry errors by providing pre-validation and row-by-row error reporting.

Architecture

Service Layer

BaseImportService (Abstract)

The "Engine" found in App\Domain\Import\Services.

• Transaction: Wraps the entire file processing loop in a single DB Transaction.
  • Implication: If Row 999 fails, Rows 1-998 are rolled back. This ensures "All or Nothing" integrity.
• Error Handling: Captures exceptions per row, adds them to an errors array, and continues (if not fatal) or aborts.
• Headers: Automatically validates standard headers against a required list.

Implementations

Service	Target	Key Validation
CustomerImportService	customers	Unique Email/Phone checks.
InventoryItemImportService	inventory_items	SKU Uniqueness. Category existence check.
VendorImportService	vendors	Tax ID format.

---

Workflows

1. CSV Upload & Process

1. UI: User downloads "Sample CSV" (generated by getSampleCsv()).
2. Upload: User uploads file.
3. Parsing:
   • Script opens file (fopen).
   • Reads Header. Validates columns.
   • Starts DB::beginTransaction().
4. Loop:
   • Reads Row.
   • Calls processRow($data).
   • If Exception: DB::rollBack(), Log Error, Stop.
   • If Validation fail: Record Error, Continue (or Rollback depending on strictness).
5. Commit: If Error Count == 0, DB::commit().
6. Report: JSON response { success: true, count: 50, failed: 0 }.

---

Audit Findings & Improvements

Strengths

• Data Integrity: The "All or Nothing" transactional approach prevents "half-imports" which are nightmare to clean up manually.
• User Guidance: Generating sample CSVs directly from code ensures the template always matches the code requirements.

Issues Identified

Major

• Memory Scalability: The importFromCsv method processes the entire file inside a SINGLE transaction while keeping the file handle open and accumulating an errors array in RAM.
  • Risk: A 50,000 row file will likely crash PHP (Memory Limit) or MySQL (Transaction Timeout).
  • Fix: Implement Chunking. Process 1000 rows, commit, then next 1000. (Trade-off: Partial imports possible).

Minor

• Validation: It uses manual if (empty($row['email'])) checks. Should rely on Laravel's Validator::make($row, rules) for cleaner, robust validation.
• Delimiter: Hardcoded as , (comma). European Excel often saves CSVs with ; (semicolon). Needs auto-detection.

---

Configuration

Config: None. Hardcoded logic.

Module Version: 1.0 Status: Production Ready (Small Files) / Beta (Large Files)