> ## Documentation Index
> Fetch the complete documentation index at: https://dhanurgo.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# CSV Deduplication

> How HostMetrics prevents duplicate records on re-import

# CSV Deduplication

When users import CSVs (manually or via Chrome extension), the system prevents duplicate records.

## Deduplication Keys

| Data Type         | Unique Key                        | Strategy                             |
| ----------------- | --------------------------------- | ------------------------------------ |
| Trip Earnings     | `(user_id, reservation_id)`       | Upsert — update existing on conflict |
| Earnings Reports  | `(user_id, composite_key)`        | Upsert — update existing on conflict |
| Toll Transactions | `(provider, date, amount, plaza)` | Skip — ignore duplicates             |

## Trip Earnings Deduplication

Each Turo trip has a unique `reservation_id`. When importing:

1. Check if a trip with the same `(user_id, reservation_id)` exists
2. If yes → **update** the existing record (trip details may change, e.g., status Completed → Paid)
3. If no → **insert** a new record
4. This is done in batch upserts of 500 rows

## Import Superseding

When a new CSV is imported:

1. A new `csv_imports` audit record is created with status `active`
2. Previous imports of the **same type** are marked as `superseded`
3. The actual trip/earning records remain — they're upserted, not replaced
4. This provides an audit trail of import history

## Chrome Extension Sync

The Chrome extension follows the same deduplication logic:

1. Extension scrapes Turo page → sends CSV content
2. Server parses CSV identically to manual upload
3. Same upsert logic applies
4. Import record is tagged with `chrome_extension` in the filename
5. The Data Sources page shows "Last synced: 2h ago" from the import record

## Re-Import Safety

It is **safe to re-import the same CSV multiple times**:

* Existing records are updated (not duplicated)
* New records are inserted
* Import count shows "X new, Y updated"
* No data loss or corruption occurs