kbe/dolibarr_shine_reconciliation
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial implementation of the reconciliation script

Browse Source 
Standalone Ruby script reconciling GoCardless payments, Dolibarr
invoices (via API), and Shine bank statements. Three-pass engine:
GC↔Dolibarr matching, open invoice audit, payout↔bank verification.
Includes dry-run and --fix mode to auto-mark Dolibarr invoices as paid.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Kevin Bataille
2026-02-26 00:23:07 +01:00
commit 4decb3cb3c
16 changed files with 19534 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
.env.example Normal file
View File

@@ -0,0 +1,12 @@
# Dolibarr API
DOLIBARR_URL=https://your-dolibarr.example.com/api/index.php
DOLIBARR_API_KEY=your_api_key
# GoCardless payment method ID for GoCardless
# Find it with: GET /setup/dictionary/payment_types
# Look for "Prélèvement GoCardless" or similar
DOLIBARR_GC_PAYMENT_ID=6
# Dolibarr bank account ID (for recording payments)
# Find it with: GET /bankaccounts
DOLIBARR_BANK_ACCOUNT_ID=1

13
.gitignore vendored Normal file
View File

@@ -0,0 +1,13 @@
# Credentials — never commit
.env
# Input data — contains customer PII and financial data
gocardless/
shine/
# Output
tmp/
# Ruby
.bundle/
vendor/bundle/

4
Gemfile Normal file
View File

@@ -0,0 +1,4 @@
source "https://rubygems.org"
gem "httparty"
gem "dotenv"

24
Gemfile.lock Normal file
View File

@@ -0,0 +1,24 @@
GEM
remote: https://rubygems.org/
specs:
bigdecimal (4.0.1)
csv (3.3.5)
dotenv (3.2.0)
httparty (0.24.2)
csv
mini_mime (>= 1.0.0)
multi_xml (>= 0.5.2)
mini_mime (1.1.5)
multi_xml (0.8.1)
bigdecimal (>= 3.1, < 5)
PLATFORMS
ruby
x86_64-linux
DEPENDENCIES
dotenv
httparty
BUNDLED WITH
2.6.7

280
README.md Normal file
View File

@@ -0,0 +1,280 @@
# Dolibarr / GoCardless / Shine Reconciliation
A standalone Ruby script that cross-checks three financial systems and flags discrepancies.
## The problem it solves
Payments flow through three separate systems that are not automatically linked:
```
Shine bank account ← GoCardless payouts ← GoCardless payments ← Dolibarr invoices
```
**Typical workflow:**
1. GoCardless initiates a direct debit for a customer
2. If the debit succeeds, you create the corresponding invoice in Dolibarr and mark it paid
3. If the debit fails, you cancel or delete the draft invoice
4. GoCardless batches collected payments into payouts and transfers them to your Shine account
Discrepancies arise when any of these steps is missed:
- A GoCardless payment succeeded but the Dolibarr invoice was never created
- A Dolibarr invoice is marked paid but no GoCardless payment can be found for it
- A GoCardless payout never appeared in the Shine bank account
---
## Requirements
- Ruby 3.x
- Bundler (`gem install bundler`)
- Network access to your Dolibarr instance
---
## Installation
```bash
cd dolibarr_shine_reconciliation
bundle install
cp .env.example .env
```
Edit `.env` and fill in your Dolibarr credentials:
```dotenv
DOLIBARR_URL=https://your-dolibarr.example.com/api/index.php
DOLIBARR_API_KEY=your_api_key
# GoCardless payment method ID in Dolibarr (used by --fix mode)
# Find it: GET /setup/dictionary/payment_types
# Look for the "Prélèvement GoCardless" entry
DOLIBARR_GC_PAYMENT_ID=6
# Bank account ID in Dolibarr (used by --fix mode)
# Find it: GET /bankaccounts
DOLIBARR_BANK_ACCOUNT_ID=1
```
---
## Exporting the data
### GoCardless — payments CSV
Dashboard → **Payments** → filter by date range → **Export CSV**
Place the file in `gocardless/`. The expected columns are:
| Column | Description |
|--------|-------------|
| `id` | Payment ID, e.g. `PM014J7X4PY98T` |
| `charge_date` | Date the customer was debited (`YYYY-MM-DD`) |
| `amount` | Amount in euros (`30.52`) |
| `description` | Free text — used as the primary match key against the Dolibarr invoice ref |
| `status` | `paid_out`, `confirmed`, `failed`, `cancelled` |
| `links.payout` | Payout ID this payment belongs to |
| `payout_date` | Date the payout was sent to your bank |
| `customers.given_name` | Customer first name |
| `customers.family_name` | Customer last name |
### Shine — bank statement CSV
App → **Comptes** → **Exporter le relevé** → select year → download
Place the annual CSV in `shine/`. The expected columns are:
| Column | Description |
|--------|-------------|
| `Date de la valeur` | Value date (`DD/MM/YYYY`) |
| `Crédit` | Credit amount in French format (`51,10`) |
| `Débit` | Debit amount |
| `Libellé` | Transaction description |
| `Nom de la contrepartie` | Counterparty name — GoCardless payouts show `GOCARDLESS SAS` here |
The Shine CSV uses semicolons as separator (`;`), UTF-8 encoding, and Windows CRLF line endings. The script handles all of this automatically.
---
## Usage
### Dry run — report only, no changes to Dolibarr
```bash
ruby bin/reconcile \
--from 2026-01-01 \
--to 2026-01-31 \
--gc gocardless/payments_export.csv \
--shine shine/Cyanet_2026-01-01_2026-12-31_EXPORT/BANQUE_2026-01-01_2026-12-31/BQ_2026-01-01_2026-12-31.csv
```
The Shine file is optional. Without it, payout verification (Pass 3) is skipped:
```bash
ruby bin/reconcile \
--from 2026-01-01 \
--to 2026-01-31 \
--gc gocardless/payments_export.csv
```
### Fix mode — auto-mark Dolibarr invoices as paid
When the script detects a GoCardless payment that was collected but the matching Dolibarr invoice is still open, `--fix` records the payment in Dolibarr via the API:
```bash
ruby bin/reconcile \
--from 2026-01-01 \
--to 2026-01-31 \
--gc gocardless/payments_export.csv \
--shine shine/.../BQ_2026-01-01_2026-12-31.csv \
--fix
```
`--fix` only affects invoices flagged `GC_PAID_DOLIBARR_OPEN`. All other entries are reported only.
---
## How matching works
The script runs three passes over the data.
### Pass 1 — GoCardless ↔ Dolibarr
For each GoCardless payment, an attempt is made to find a matching Dolibarr invoice in two steps:
**Strong match** — the GoCardless `description` field equals the Dolibarr invoice `ref` exactly (case-insensitive). This fires when you put the invoice reference in the GoCardless payment description at creation time.
**Soft match** — if no strong match is found, the script looks for a Dolibarr invoice where:
- The amount is identical (compared in cents to avoid floating-point errors)
- The invoice date is within 7 days of the GoCardless `charge_date`
- The customer name on the Dolibarr invoice matches the GoCardless customer name (accent-insensitive, word-order-insensitive)
Once matched (or not), each payment is assigned one of these flags:
| Flag | Meaning | Action |
|------|---------|--------|
| `MATCHED` | GC payment collected, Dolibarr invoice paid | None |
| `GC_PAID_DOLIBARR_OPEN` | GC collected but Dolibarr invoice is still open | Create the invoice / use `--fix` |
| `GC_NO_INVOICE` | GC payment collected, no Dolibarr invoice found at all | Create the invoice in Dolibarr |
| `GC_FAILED` | GC payment failed | Check if Dolibarr invoice was correctly cancelled |
| `GC_CANCELLED` | GC payment was cancelled before collection | No action |
| `DOLIBARR_PAID_NO_GC` | Dolibarr invoice paid (in the date range), no GC payment found | Verify — may be a manual or cash payment |
After processing all GC payments, open Dolibarr invoices with no GC counterpart are flagged:
| Flag | Meaning | Action |
|------|---------|--------|
| `DOLIBARR_OPEN_NO_GC` | Dolibarr invoice open, no GC payment found | Follow up — missed debit or GC export is incomplete |
### Pass 2 — open Dolibarr invoice audit
All invoices fetched from Dolibarr with status `open` (validated, not yet paid) that were not matched by any GC payment are listed as `DOLIBARR_OPEN_NO_GC`. Overdue invoices (due date in the past) are highlighted.
### Pass 3 — GoCardless payouts ↔ Shine bank
GoCardless batches individual payments into payouts and transfers them as a single bank credit. The script groups `paid_out` payments by their payout ID, sums the amounts, and looks for a matching credit in Shine:
1. **Exact match** — same amount, date within 2 days → `PAYOUT_VERIFIED`
2. **Date match only** — date within 2 days but amount differs → `PAYOUT_AMOUNT_MISMATCH` (expected: GoCardless deducts its fee from the payout, so the bank credit is always slightly less than the sum of payments)
3. **No match found** → `PAYOUT_MISSING`
`PAYOUT_AMOUNT_MISMATCH` is the normal case when GoCardless fees are deducted. The difference shown in the report is the total fee charged for the period.
---
## Output
### Terminal report
```
============================================================
RECONCILIATION REPORT: 2026-01-01 to 2026-01-31
============================================================
DOLIBARR
Total invoices in scope: 9
Open (no GC match): 2 ← needs attention
Paid (GC matched): 3
GOCARDLESS ↔ DOLIBARR
Matched (paid both sides): 3 ✓
GC paid / Dolibarr open: 0
Dolibarr paid / no GC: 0
GC failed: 1
GC cancelled: 0
GC payment / no invoice: 4 ← investigate
SHINE ↔ GOCARDLESS PAYOUTS
Payouts expected: 2
Verified: 0
Amount mismatch: 2 ← check GC fees
Missing in Shine: 0
Expected total: €107.74
Actual total: €104.91
Difference: €-2.83 ← GoCardless fees
ACTIONS NEEDED (6)
------------------------------------------------------------
1. [DOLIBARR_OPEN_NO_GC] FA2502-0075 €29.44 ARTHUR Muriel overdue since 2025-02-01
2. [GC_NO_INVOICE] GC: PM01RE90... €26.10 MARIE RIVIERE 2026-01-05
...
```
### CSV export
A file `tmp/reconciliation_YYYY-MM-DD.csv` is written after every run with one row per invoice/payment, including the flag and recommended action. Suitable for importing into a spreadsheet for manual review.
---
## Project structure
```
bin/reconcile Entry point — parses CLI arguments, orchestrates the run
lib/
boot.rb Loads all dependencies
dolibarr/
client.rb HTTP client for the Dolibarr REST API (HTTParty)
reconciliation/
dolibarr_fetcher.rb Fetches invoices and customer names via Dolibarr API
gocardless_parser.rb Parses the GoCardless payments CSV
shine_parser.rb Parses the Shine bank statement CSV
engine.rb 3-pass matching logic, produces flagged result set
reporter.rb Formats and prints the terminal report, writes CSV
fixer.rb Calls Dolibarr API to record payments (--fix mode)
gocardless/ Drop GoCardless CSV exports here
shine/ Shine annual export directories (as downloaded)
tmp/ Output CSVs written here
.env.example Environment variable template
docs/
reconciliation_plan.md Original design document
dolibarr.json Dolibarr Swagger API spec
```
---
## Dolibarr API notes
The script uses the Dolibarr REST API with an API key (`DOLAPIKEY` header). Key endpoints:
| Method | Path | Purpose |
|--------|------|---------|
| `GET` | `/invoices?status=1` | Fetch all non-draft invoices (open and paid) |
| `GET` | `/thirdparties` | Fetch customer names for invoice matching |
| `POST` | `/invoices/paymentsdistributed` | Record a payment against an invoice (`--fix`) |
| `GET` | `/setup/dictionary/payment_types` | Look up the GoCardless payment method ID |
| `GET` | `/bankaccounts` | Look up the bank account ID |
The `status=1` query in Dolibarr returns all non-draft invoices regardless of payment state. The script uses the `statut` field in the response (`1`=open, `2`=paid, `3`=cancelled) to distinguish them. Cancelled invoices are excluded from reconciliation.
---
## Limitations and known behaviour
**GoCardless fee deductions** — Payout amounts in Shine are always slightly less than the sum of the underlying payments because GoCardless deducts its transaction fee from the payout. This is expected and reported as `PAYOUT_AMOUNT_MISMATCH`, not an error.
**Incomplete GoCardless export** — If your CSV export does not cover the full date range, payments from outside the export window will cause open Dolibarr invoices to appear as `DOLIBARR_OPEN_NO_GC`. Export all payments for the period you are reconciling.
**Customer name matching** — The soft match normalises names by stripping accents, lowercasing, and sorting words, so "DUPONT Jean" matches "Jean Dupont". If a customer's name is spelled differently in GoCardless vs Dolibarr, the soft match will fail and the payment will appear as `GC_NO_INVOICE`. Correct the name in one of the systems and rerun.
**Credit notes** — Dolibarr credit notes (`AV...` prefix) with negative amounts are included in the invoice fetch and will appear as `DOLIBARR_PAID_NO_GC` if they fall in the reconciliation period with no corresponding GoCardless refund. This is normal — credit notes are typically settled internally, not via GoCardless.
**Supplier invoices** — Dolibarr supplier invoices (`/supplierinvoices` endpoint) are on a completely separate API path and are never fetched or considered by this script.

88
bin/reconcile Executable file
View File

@@ -0,0 +1,88 @@
#!/usr/bin/env ruby
# frozen_string_literal: true
require "optparse"
require "date"
require_relative "../lib/boot"
options = {
fix: false,
from: nil,
to: nil,
gc: nil,
shine: nil
}
OptionParser.new do |opts|
opts.banner = <<~BANNER
Usage: bin/reconcile --from DATE --to DATE --gc PATH [--shine PATH] [--fix]
Options:
BANNER
opts.on("--from DATE", "Start date (YYYY-MM-DD), inclusive") do |v|
options[:from] = Date.parse(v)
end
opts.on("--to DATE", "End date (YYYY-MM-DD), inclusive") do |v|
options[:to] = Date.parse(v)
end
opts.on("--gc PATH", "GoCardless payments CSV file path") do |v|
options[:gc] = v
end
opts.on("--shine PATH", "Shine bank statement CSV file path (optional)") do |v|
options[:shine] = v
end
opts.on("--fix", "Apply fixes: mark matching Dolibarr invoices as paid via API") do
options[:fix] = true
end
opts.on("-h", "--help", "Show this help") do
puts opts
exit
end
end.parse!
# Validate required options
errors = []
errors << "--from is required" unless options[:from]
errors << "--to is required" unless options[:to]
errors << "--gc is required" unless options[:gc]
errors << "--gc file not found: #{options[:gc]}" if options[:gc] && !File.exist?(options[:gc])
errors << "--shine file not found: #{options[:shine]}" if options[:shine] && !File.exist?(options[:shine])
unless errors.empty?
errors.each { |e| warn "Error: #{e}" }
exit 1
end
# Run reconciliation
client = Dolibarr::Client.new
fetcher = Reconciliation::DolibarrFetcher.new(client, from: options[:from], to: options[:to])
gc_data = Reconciliation::GocardlessParser.parse(options[:gc], from: options[:from], to: options[:to])
shine_data = options[:shine] ? Reconciliation::ShineParser.parse(options[:shine]) : []
dolibarr_invoices = fetcher.fetch_invoices
engine = Reconciliation::Engine.new(
dolibarr_invoices: dolibarr_invoices,
gc_payments: gc_data,
shine_transactions: shine_data,
from: options[:from],
to: options[:to]
)
result = engine.run
reporter = Reconciliation::Reporter.new(result, from: options[:from], to: options[:to])
reporter.print_summary
if options[:fix]
fixer = Reconciliation::Fixer.new(client)
fixer.apply(result[:gc_paid_dolibarr_open])
end
reporter.write_csv

18004
docs/dolibarr.json Normal file
View File

File diff suppressed because it is too large Load Diff

269
docs/reconciliation_plan.md Normal file
View File

@@ -0,0 +1,269 @@
# Invoice Reconciliation — Design & Implementation Plan
## Context
**Stack:** Ruby (Rails or standalone script)
**Purpose:** Reconcile GoCardless payments, Shine bank statements, and Dolibarr invoices to ensure all three systems are in sync.
**Billing flow:** Customers are billed manually via GoCardless direct debit. Invoices are managed in Dolibarr. Cash lands in a Shine business bank account.
---
## The Problem
Three systems hold overlapping financial data but are not automatically linked:
```
Shine bank ◄─── GoCardless payouts ◄─── GoCardless payments ◄─── Dolibarr invoices
```
Discrepancies arise when:
- A GoCardless payment is collected but Dolibarr is not marked as paid
- A Dolibarr invoice is marked paid but no GoCardless payment exists
- A GoCardless payout (batch) never landed in the Shine account
---
## Data Sources
| Source | What it contains | How to export |
|--------|-----------------|---------------|
| **Dolibarr API** | Invoices, payment status, customer info | Fetched live via REST API (`/invoices`) |
| **GoCardless payments CSV** | Individual payment ID, amount, date, status, customer | Dashboard → Payments → Export CSV |
| **GoCardless payouts CSV** | Payout ID, total amount, arrival date (batched bank transfers) | Dashboard → Payouts → Export CSV |
| **Shine bank CSV** | Bank transactions: date, description, amount, balance | App → Comptes → Exporter le relevé |
---
## Architecture
```
lib/tasks/reconcile.rake ← CLI entry point (rake task)
app/services/reconciliation/
├── dolibarr_fetcher.rb ← Fetch invoices + payment records via Dolibarr API
├── gocardless_parser.rb ← Parse GoCardless payments CSV
├── gocardless_payout_parser.rb ← Parse GoCardless payouts CSV
├── shine_parser.rb ← Parse Shine bank statement CSV
├── engine.rb ← Core matching logic (3 passes)
└── reporter.rb ← Terminal summary + CSV output
```
---
## Reconciliation Passes
### Pass 1 — Dolibarr open invoice audit
- Fetch all Dolibarr invoices with `status=1` (validated/open) for the date range
- Flag any that are past their due date: these should have a GoCardless payment but don't
- Output: list of overdue open invoices needing action
### Pass 2 — GoCardless ↔ Dolibarr matching
For each GoCardless payment in the CSV:
**Matching strategy (in order of confidence):**
1. **Strong match**: `num_payment` field in Dolibarr payment record == GoCardless payment ID
2. **Soft match**: `amount_ttc` + `customer_name` + date within ±5 days
**Flags:**
- `GC_PAID_DOLIBARR_OPEN` — GoCardless collected payment, Dolibarr invoice still open → needs `mark_as_paid` call
- `DOLIBARR_PAID_NO_GC` — Dolibarr marked paid but no GoCardless payment found → manual payment, verify
- `GC_FAILED` — GoCardless payment failed, check if Dolibarr invoice is still open
- `MATCHED` — Both sides agree, no action needed
### Pass 3 — Shine bank ↔ GoCardless payout verification
- GoCardless batches individual payments into payouts and deposits them as a single bank transfer
- Each GoCardless payout should appear as a credit in Shine (description contains "GOCARDLESS" or similar)
- Match by: payout arrival date ±2 days + amount
**Flags:**
- `PAYOUT_MISSING` — GoCardless payout not found in Shine bank statement
- `AMOUNT_MISMATCH` — Payout found but amounts differ (possible fee deduction issue)
- `VERIFIED` — Payout confirmed in bank account
---
## Dolibarr API Endpoints Used
```
GET /invoices?status=1&limit=500 # Open/validated invoices
GET /invoices?status=2&limit=500 # Paid invoices
GET /invoices/{id} # Single invoice with payment history
GET /invoices/{id}/payments # Payment records for an invoice
POST /invoices/paymentsdistributed # Record a payment (for --fix mode)
```
**Key Dolibarr invoice fields:**
- `ref` — Invoice reference (e.g., `FA2601-0034`)
- `total_ttc` — Total amount including tax (euros, float)
- `paye` — Payment flag: `1` = paid, `0` = unpaid
- `sumpayed` — Total amount already paid
- `remaintopay` — Remaining balance due
- `socid` — Customer (third-party) ID
- `status` — `0`=draft, `1`=validated, `2`=paid, `3`=cancelled
**Dolibarr mark-as-paid payload:**
```json
{
"arrayofamounts": {
"{INVOICE_ID}": {
"amount": 19.99,
"multicurrency_amount": null
}
},
"datepaye": 1748524447,
"paymentid": 6,
"closepaidinvoices": "yes",
"accountid": 1,
"num_payment": "PM01234567",
"comment": "GoCardless payment — auto-reconciled"
}
```
(`paymentid: 6` = GoCardless payment method ID in Dolibarr — verify this value in your instance)
---
## GoCardless CSV Format
GoCardless payment export columns (verify against your actual export):
```
id, amount, currency, status, charge_date, description, reference,
customer_id, customer_name, mandate_id, payout_id
```
Relevant statuses: `paid_out`, `confirmed`, `failed`, `cancelled`, `pending_submission`
GoCardless payout export columns:
```
id, amount, currency, status, arrival_date, reference
```
---
## Shine CSV Format
Shine bank export columns (verify against your actual export):
```
Date, Libellé, Montant, Catégorie, Notes, Solde
```
GoCardless payouts typically appear with `Libellé` containing `GOCARDLESS` or `GC`.
**Note:** Column names may vary by export language/version. Make the parser configurable.
---
## CLI Interface
```bash
# Dry run — report only, no changes
bin/rails reconcile:run \
FROM=2026-01-01 \
TO=2026-01-31 \
GC=tmp/gocardless_payments.csv \
GC_PAYOUTS=tmp/gocardless_payouts.csv \
SHINE=tmp/shine_january.csv
# Auto-fix — mark Dolibarr invoices as paid where GC payment is confirmed
bin/rails reconcile:run \
FROM=2026-01-01 \
TO=2026-01-31 \
GC=tmp/gocardless_payments.csv \
GC_PAYOUTS=tmp/gocardless_payouts.csv \
SHINE=tmp/shine_january.csv \
FIX=true
```
---
## Expected Report Output
```
=== RECONCILIATION REPORT: 2026-01-01 to 2026-01-31 ===
DOLIBARR SUMMARY
Total invoices validated: 47
Total invoices paid: 43
Open and overdue: 4 ← ACTION NEEDED
GOCARDLESS ↔ DOLIBARR
Matched (no action): 43 ✓
GC paid / Dolibarr open: 3 ← Dolibarr needs marking as paid
Dolibarr paid / no GC: 1 ← Verify manual payment
GC failed: 0
SHINE ↔ GOCARDLESS PAYOUTS
Payouts expected: 8
Payouts found in Shine: 8 ✓
Total amount expected: €842.58
Total amount received: €842.58
Difference: €0.00 ✓
ACTIONS NEEDED:
1. [GC_PAID_DOLIBARR_OPEN] FA2601-0034 €19.99 DUPONT Jean GC: PM01234567 2026-01-05
2. [GC_PAID_DOLIBARR_OPEN] FA2601-0041 €29.99 MARTIN Paul GC: PM05678901 2026-01-12
3. [GC_PAID_DOLIBARR_OPEN] FA2601-0052 €19.99 DURAND Marie GC: PM09012345 2026-01-19
4. [DOLIBARR_PAID_NO_GC] FA2601-0023 €19.99 LEROY Claude Paid: 2026-01-08 (no GC match)
Report saved to: tmp/reconciliation_2026-01-31.csv
```
---
## Output CSV Format
```csv
invoice_ref,customer_name,amount_ttc,invoice_date,dolibarr_status,gc_payment_id,gc_status,gc_charge_date,match_status,action
FA2601-0034,DUPONT Jean,19.99,2026-01-01,open,PM01234567,paid_out,2026-01-05,GC_PAID_DOLIBARR_OPEN,mark_dolibarr_paid
FA2601-0041,MARTIN Paul,29.99,2026-01-08,open,PM05678901,paid_out,2026-01-12,GC_PAID_DOLIBARR_OPEN,mark_dolibarr_paid
FA2601-0023,LEROY Claude,19.99,2026-01-07,paid,,,,DOLIBARR_PAID_NO_GC,verify_manually
FA2601-0012,DUPONT Jean,19.99,2025-12-01,paid,PM98765432,paid_out,2025-12-05,MATCHED,none
```
---
## Implementation Notes
### Matching tolerance
- **Amount**: exact match (compare in cents to avoid float issues — multiply Dolibarr's euro amounts by 100)
- **Date**: ±5 days between GoCardless `charge_date` and Dolibarr invoice `date`
- **Customer**: normalize names before comparing (strip accents, lowercase, trim)
### Dolibarr client setup
The `Dolibarr::Client` requires:
```
DOLIBARR_URL=https://your-dolibarr.example.com/api/index.php
DOLIBARR_API_KEY=your_api_key
```
### Edge cases to handle
- One GoCardless payment covering multiple invoices (rare but possible)
- Credit notes in Dolibarr offsetting invoice balances
- GoCardless payment retried after initial failure (same invoice, multiple GC payment IDs)
- Shine CSV encoding (often ISO-8859-1, convert to UTF-8)
- GoCardless fees: payouts may be slightly less than sum of payments due to GC fees
### Dolibarr payment method ID
When calling `paymentsdistributed`, `paymentid` must match the Dolibarr payment method ID for GoCardless in your instance. Check:
```
GET /setup/dictionary/payment_types
```
Find the entry for GoCardless (often named "Prélèvement GoCardless" or similar).
---
## Future Enhancements
- **Scheduled run**: Run monthly via cron, email report to admin
- **Web interface**: Upload CSVs via admin UI, view reconciliation in browser
- **Stripe support**: Same engine, add a `StripeParser` for Stripe payouts
- **Webhook-driven**: Instead of CSV imports, consume GoCardless webhooks in real time to auto-reconcile as payments land

12
lib/boot.rb Normal file
View File

@@ -0,0 +1,12 @@
# frozen_string_literal: true
require "bundler/setup"
require "dotenv/load"
require_relative "dolibarr/client"
require_relative "reconciliation/dolibarr_fetcher"
require_relative "reconciliation/gocardless_parser"
require_relative "reconciliation/shine_parser"
require_relative "reconciliation/engine"
require_relative "reconciliation/reporter"
require_relative "reconciliation/fixer"

83
lib/dolibarr/client.rb Normal file
View File

@@ -0,0 +1,83 @@
# frozen_string_literal: true
require "httparty"
require "json"
module Dolibarr
class Client
include HTTParty
class NotFoundError < StandardError; end
class ValidationError < StandardError; end
class Error < StandardError; end
def initialize(
base_url: ENV.fetch("DOLIBARR_URL"),
api_key: ENV.fetch("DOLIBARR_API_KEY")
)
@base_url = base_url
@headers = {
"DOLAPIKEY" => api_key,
"Content-Type" => "application/json",
"Accept" => "application/json"
}
end
def get(path, params = {})
perform_request(:get, path, params)
end
def post(path, data = {})
perform_request(:post, path, data)
end
def put(path, data = {})
perform_request(:put, path, data)
end
private
def perform_request(method, path, data = {})
url = "#{@base_url}#{path}"
opts = { headers: @headers }
case method
when :get
opts[:query] = data unless data.empty?
response = self.class.get(url, opts)
when :post
opts[:body] = data.to_json
response = self.class.post(url, opts)
when :put
opts[:body] = data.to_json
response = self.class.put(url, opts)
end
handle_response(response)
rescue => e
raise Error, "Connection error: #{e.message}"
end
def handle_response(response)
return nil if response.body.nil? || response.body.strip.empty?
code = response.code.to_i
case code
when 200..299
response.parsed_response
when 404
raise NotFoundError, "Not found (404)"
when 400
raise ValidationError, "Validation error: #{response.body}"
when 401
raise Error, "Authentication failed (401) — check DOLIBARR_API_KEY"
when 500..599
raise Error, "Dolibarr server error (#{code}): #{response.body}"
else
raise Error, "Unexpected status #{code}: #{response.body}"
end
rescue JSON::ParserError => e
raise Error, "JSON parse error: #{e.message}"
end
end
end

115
lib/reconciliation/dolibarr_fetcher.rb Normal file
View File

@@ -0,0 +1,115 @@
# frozen_string_literal: true
require "date"
module Reconciliation
class DolibarrFetcher
# Dolibarr invoice statuses
STATUS_DRAFT = 0
STATUS_VALIDATED = 1
STATUS_PAID = 2
STATUS_CANCELLED = 3
Invoice = Struct.new(
:id, # Dolibarr internal ID
:ref, # Invoice reference, e.g. "FA2407-0003"
:status, # 0=draft, 1=validated/open, 2=paid, 3=cancelled
:total_ttc, # Total amount TTC (euros, float)
:amount_cents, # total_ttc * 100, rounded (integer)
:paye, # 0=unpaid, 1=paid
:sumpayed, # Amount already paid (euros)
:remain_to_pay, # Remaining balance (euros)
:date, # Invoice date (Date)
:due_date, # Due date (Date)
:customer_id, # socid
:customer_name, # nom of third party if present
keyword_init: true
)
def initialize(client, from:, to:)
@client = client
@from = from
@to = to
end
# Fetches all validated invoices and resolves customer names.
# Note: querying status=1 returns all non-draft invoices in Dolibarr (open, paid, cancelled).
# Cancelled invoices are filtered out here.
def fetch_invoices
$stderr.puts "[DolibarrFetcher] Fetching thirdparty names..."
name_by_id = fetch_thirdparty_names
$stderr.puts "[DolibarrFetcher] Fetching invoices..."
raw_invoices = fetch_all(status: STATUS_VALIDATED)
invoices = raw_invoices
.map { |raw| parse_invoice(raw, name_by_id) }
.reject { |inv| inv.status == STATUS_CANCELLED }
open_count = invoices.count { |inv| inv.status == STATUS_VALIDATED }
paid_count = invoices.count { |inv| inv.status == STATUS_PAID }
$stderr.puts "[DolibarrFetcher] Total: #{invoices.size} invoices (#{open_count} open, #{paid_count} paid)"
invoices
rescue Dolibarr::Client::Error => e
$stderr.puts "[DolibarrFetcher] Error: #{e.message}"
raise
end
private
def fetch_all(status:, sqlfilters: nil)
invoices = []
page = 0
loop do
params = { status: status, limit: 100, page: page }
params[:sqlfilters] = sqlfilters if sqlfilters
batch = @client.get("/invoices", params)
break if batch.nil? || !batch.is_a?(Array) || batch.empty?
invoices.concat(batch)
break if batch.size < 100
page += 1
end
invoices
end
def parse_invoice(raw, name_by_id = {})
customer_id = raw["socid"].to_i
Invoice.new(
id: raw["id"].to_i,
ref: raw["ref"].to_s.strip,
status: raw["statut"].to_i,
total_ttc: raw["total_ttc"].to_f,
amount_cents: (raw["total_ttc"].to_f * 100).round,
paye: raw["paye"].to_i,
sumpayed: raw["sumpayed"].to_f,
remain_to_pay: raw["remaintopay"].to_f,
date: parse_unix_date(raw["datef"] || raw["date"]),
due_date: parse_unix_date(raw["date_lim_reglement"]),
customer_id: customer_id,
customer_name: name_by_id[customer_id] || raw.dig("thirdparty", "name").to_s.strip
)
end
def fetch_thirdparty_names
result = {}
page = 0
loop do
batch = @client.get("/thirdparties", limit: 100, page: page)
break if batch.nil? || !batch.is_a?(Array) || batch.empty?
batch.each { |tp| result[tp["id"].to_i] = tp["name"].to_s.strip }
break if batch.size < 100
page += 1
end
result
end
def parse_unix_date(value)
return nil if value.nil? || value.to_s.strip.empty? || value.to_i.zero?
Time.at(value.to_i).utc.to_date
rescue
nil
end
end
end

243
lib/reconciliation/engine.rb Normal file
View File

@@ -0,0 +1,243 @@
# frozen_string_literal: true
require "date"
module Reconciliation
class Engine
# Match flags
MATCHED = :matched # GC paid + Dolibarr paid — all good
GC_PAID_DOLIBARR_OPEN = :gc_paid_dolibarr_open # GC collected, Dolibarr still open — FIX NEEDED
GC_FAILED = :gc_failed # GC payment failed
GC_CANCELLED = :gc_cancelled # GC payment cancelled (no action needed)
GC_NO_INVOICE = :gc_no_invoice # GC payment with no Dolibarr invoice found
DOLIBARR_PAID_NO_GC = :dolibarr_paid_no_gc # Dolibarr paid, no GC match — verify manually
DOLIBARR_OPEN_NO_GC = :dolibarr_open_no_gc # Dolibarr open, no GC payment found (overdue)
# Payout flags
PAYOUT_VERIFIED = :payout_verified
PAYOUT_MISSING = :payout_missing
PAYOUT_AMOUNT_MISMATCH = :payout_amount_mismatch
DATE_TOLERANCE = 7 # days for GC charge_date ↔ Dolibarr invoice date soft match
PAYOUT_DATE_TOLERANCE = 2 # days for GC payout_date ↔ Shine credit date
# Result structs
Match = Struct.new(
:flag,
:invoice, # DolibarrFetcher::Invoice or nil
:payment, # GocardlessParser::Payment or nil
:match_type, # :strong, :soft, or nil
keyword_init: true
)
PayoutMatch = Struct.new(
:flag,
:payout_id,
:payout_date,
:expected_amount_cents,
:shine_transaction, # ShineParser::Transaction or nil
:actual_amount_cents,
:gc_payment_ids,
keyword_init: true
)
def initialize(dolibarr_invoices:, gc_payments:, shine_transactions:, from:, to:)
@invoices = dolibarr_invoices
@gc_payments = gc_payments
@shine = shine_transactions
@from = from
@to = to
end
def run
matches = pass1_gc_vs_dolibarr
overdue = pass2_overdue_open_invoices(matches)
payout_matches = pass3_payouts_vs_shine
{
matches: matches,
matched: matches.select { |m| m.flag == MATCHED },
gc_paid_dolibarr_open: matches.select { |m| m.flag == GC_PAID_DOLIBARR_OPEN },
gc_failed: matches.select { |m| m.flag == GC_FAILED },
gc_cancelled: matches.select { |m| m.flag == GC_CANCELLED },
gc_no_invoice: matches.select { |m| m.flag == GC_NO_INVOICE },
dolibarr_paid_no_gc: matches.select { |m| m.flag == DOLIBARR_PAID_NO_GC },
dolibarr_open_no_gc: overdue,
payout_matches: payout_matches
}
end
private
# -------------------------------------------------------------------------
# Pass 1 — Match each GoCardless payment to a Dolibarr invoice
# -------------------------------------------------------------------------
def pass1_gc_vs_dolibarr
matched_invoice_ids = []
results = []
@gc_payments.each do |payment|
invoice, match_type = find_invoice(payment)
if invoice
matched_invoice_ids << invoice.id
end
flag = determine_flag(payment, invoice)
results << Match.new(flag: flag, invoice: invoice, payment: payment, match_type: match_type)
end
# Dolibarr paid invoices in the date range with no GC match.
# Scoped to the date range to avoid flagging every historical paid invoice.
matched_set = matched_invoice_ids.to_set
@invoices.each do |inv|
next if matched_set.include?(inv.id)
next unless inv.status == DolibarrFetcher::STATUS_PAID
# Only flag paid invoices whose date falls within the reconciliation window
next if inv.date && (inv.date < @from || inv.date > @to)
results << Match.new(
flag: DOLIBARR_PAID_NO_GC,
invoice: inv,
payment: nil,
match_type: nil
)
end
results
end
# -------------------------------------------------------------------------
# Pass 2 — Flag open Dolibarr invoices with no GC match that are overdue
# -------------------------------------------------------------------------
def pass2_overdue_open_invoices(matches)
matched_invoice_ids = matches.filter_map { |m| m.invoice&.id }.to_set
@invoices.select do |inv|
inv.status == DolibarrFetcher::STATUS_VALIDATED &&
!matched_invoice_ids.include?(inv.id)
end
end
# -------------------------------------------------------------------------
# Pass 3 — Match GoCardless payouts to Shine bank credits
# -------------------------------------------------------------------------
def pass3_payouts_vs_shine
return [] if @shine.empty?
gc_credits = ShineParser.gocardless_credits(@shine)
# Group paid_out payments by payout_id.
# Only paid_out payments are included in a payout; failed/cancelled are not.
by_payout = @gc_payments
.select { |p| p.status == "paid_out" && p.payout_id && !p.payout_id.empty? && p.payout_date }
.group_by(&:payout_id)
used_shine_ids = []
by_payout.filter_map do |payout_id, payments|
payout_date = payments.map(&:payout_date).compact.max
expected_cents = payments.sum(&:amount_cents)
shine_tx = gc_credits.find do |tx|
!used_shine_ids.include?(tx.id) &&
(tx.date - payout_date).abs <= PAYOUT_DATE_TOLERANCE &&
tx.credit_cents == expected_cents
end
if shine_tx
used_shine_ids << shine_tx.id
flag = PAYOUT_VERIFIED
actual = shine_tx.credit_cents
else
# Try date match only (amount mismatch — possible GC fees)
shine_tx = gc_credits.find do |tx|
!used_shine_ids.include?(tx.id) &&
(tx.date - payout_date).abs <= PAYOUT_DATE_TOLERANCE
end
if shine_tx
used_shine_ids << shine_tx.id
flag = PAYOUT_AMOUNT_MISMATCH
actual = shine_tx.credit_cents
else
flag = PAYOUT_MISSING
actual = 0
end
end
PayoutMatch.new(
flag: flag,
payout_id: payout_id,
payout_date: payout_date,
expected_amount_cents: expected_cents,
shine_transaction: shine_tx,
actual_amount_cents: actual,
gc_payment_ids: payments.map(&:id)
)
end
end
# -------------------------------------------------------------------------
# Invoice lookup
# -------------------------------------------------------------------------
def find_invoice(payment)
# 1. Strong match: GC description == Dolibarr invoice ref
# Applies when the user puts the invoice ref in the GC payment description.
invoice = @invoices.find do |inv|
!payment.description.empty? &&
inv.ref.casecmp(payment.description) == 0
end
return [invoice, :strong] if invoice
# 2. Soft match: same amount + customer name + date within tolerance.
# The user creates invoices after GC payment succeeds, so the invoice date
# should be close to the GC charge_date. Since multiple customers pay the
# same amount, customer name matching is required to avoid false positives.
invoice = @invoices.find do |inv|
inv.amount_cents == payment.amount_cents &&
inv.date &&
(inv.date - payment.charge_date).abs <= DATE_TOLERANCE &&
names_match?(inv.customer_name, payment.customer_name)
end
return [invoice, :soft] if invoice
[nil, nil]
end
# Normalise a name for comparison: remove accents, lowercase, collapse spaces.
def normalize_name(name)
name.to_s
.unicode_normalize(:nfd)
.gsub(/\p{Mn}/, "") # strip combining diacritical marks
.downcase
.gsub(/[^a-z0-9 ]/, "")
.split
.sort
.join(" ")
end
def names_match?(dolibarr_name, gc_name)
return false if dolibarr_name.to_s.strip.empty? || gc_name.to_s.strip.empty?
# Both names normalised and compared as sorted word sets so
# "DUPONT Jean" matches "Jean DUPONT" and accent variants.
normalize_name(dolibarr_name) == normalize_name(gc_name)
end
def determine_flag(payment, invoice)
if GocardlessParser::COLLECTED_STATUSES.include?(payment.status)
if invoice.nil?
GC_NO_INVOICE
elsif invoice.status == DolibarrFetcher::STATUS_PAID
MATCHED
else
GC_PAID_DOLIBARR_OPEN
end
elsif GocardlessParser::FAILED_STATUSES.include?(payment.status)
GC_FAILED
else
GC_CANCELLED
end
end
end
end

56
lib/reconciliation/fixer.rb Normal file
View File

@@ -0,0 +1,56 @@
# frozen_string_literal: true
module Reconciliation
# Applies fixes to Dolibarr: marks invoices as paid using GoCardless payment data.
class Fixer
def initialize(client)
@client = client
end
# Accepts an array of Engine::Match objects with flag GC_PAID_DOLIBARR_OPEN
def apply(matches)
return if matches.empty?
gc_payment_id = ENV.fetch("DOLIBARR_GC_PAYMENT_ID", nil)&.to_i
bank_id = ENV.fetch("DOLIBARR_BANK_ACCOUNT_ID", "1").to_i
if gc_payment_id.nil?
warn "[Fixer] DOLIBARR_GC_PAYMENT_ID not set in .env — cannot fix. " \
"Run: GET /setup/dictionary/payment_types to find the GoCardless payment type ID."
return
end
puts ""
puts "APPLYING FIXES (#{matches.size} invoices)..."
matches.each do |m|
inv = m.invoice
pay = m.payment
payload = {
arrayofamounts: {
inv.id.to_s => {
amount: "%.2f" % (inv.amount_cents / 100.0),
multicurrency_amount: nil
}
},
datepaye: pay.charge_date.to_time.to_i,
paymentid: gc_payment_id,
closepaidinvoices: "yes",
accountid: bank_id,
num_payment: pay.id,
comment: "GoCardless payment — auto-reconciled (#{pay.charge_date})"
}
begin
@client.post("/invoices/paymentsdistributed", payload)
puts " ✓ Marked #{inv.ref} as paid (GC: #{pay.id})"
rescue Dolibarr::Client::ValidationError => e
puts " ✗ #{inv.ref} — validation error: #{e.message}"
rescue Dolibarr::Client::Error => e
puts " ✗ #{inv.ref} — API error: #{e.message}"
end
end
end
end
end

64
lib/reconciliation/gocardless_parser.rb Normal file
View File

@@ -0,0 +1,64 @@
# frozen_string_literal: true
require "csv"
require "date"
module Reconciliation
class GocardlessParser
Payment = Struct.new(
:id, # e.g. "PM014J7X4PY98T"
:charge_date, # Date
:amount_cents, # Integer (euros * 100)
:description, # Free text — often the Dolibarr invoice ref
:status, # "paid_out", "failed", "cancelled", etc.
:payout_id, # e.g. "PO0010R1ARR0QZ"
:payout_date, # Date or nil
:customer_name, # "Jean DUPONT"
:customer_email,
keyword_init: true
)
# Statuses that represent a successful collection from the customer.
COLLECTED_STATUSES = %w[paid_out confirmed].freeze
FAILED_STATUSES = %w[failed].freeze
def self.parse(csv_path, from: nil, to: nil)
rows = CSV.read(csv_path, headers: true, encoding: "UTF-8")
payments = rows.filter_map do |row|
charge_date = safe_parse_date(row["charge_date"])
next if charge_date.nil?
next if from && charge_date < from
next if to && charge_date > to
payout_date = safe_parse_date(row["payout_date"])
Payment.new(
id: row["id"].to_s.strip,
charge_date: charge_date,
amount_cents: (row["amount"].to_f * 100).round,
description: row["description"].to_s.strip,
status: row["status"].to_s.strip,
payout_id: row["links.payout"].to_s.strip.then { |v| v.empty? ? nil : v },
payout_date: payout_date,
customer_name: build_name(row["customers.given_name"], row["customers.family_name"]),
customer_email: row["customers.email"].to_s.strip
)
end
$stderr.puts "[GocardlessParser] Loaded #{payments.size} payments from #{csv_path}"
payments
end
private_class_method def self.safe_parse_date(str)
return nil if str.nil? || str.strip.empty?
Date.parse(str.strip)
rescue Date::Error, ArgumentError
nil
end
private_class_method def self.build_name(given, family)
[given.to_s.strip, family.to_s.strip].reject(&:empty?).join(" ")
end
end
end

203
lib/reconciliation/reporter.rb Normal file
View File

@@ -0,0 +1,203 @@
# frozen_string_literal: true
require "csv"
require "date"
module Reconciliation
class Reporter
def initialize(result, from:, to:)
@result = result
@from = from
@to = to
end
def print_summary
r = @result
puts ""
puts "=" * 60
puts " RECONCILIATION REPORT: #{@from} to #{@to}"
puts "=" * 60
# --- Dolibarr overview ---
total_invoices = r[:matches].filter_map(&:invoice).uniq(&:id).size +
r[:dolibarr_open_no_gc].size
open_count = r[:dolibarr_open_no_gc].size +
r[:gc_paid_dolibarr_open].size
puts ""
puts "DOLIBARR"
puts " Total invoices in scope: #{total_invoices}"
puts " Open (no GC match): #{r[:dolibarr_open_no_gc].size}#{r[:dolibarr_open_no_gc].any? ? ' ← needs attention' : ''}"
puts " Paid (GC matched): #{r[:matched].size}"
# --- GoCardless ↔ Dolibarr ---
puts ""
puts "GOCARDLESS ↔ DOLIBARR"
puts " Matched (paid both sides): #{r[:matched].size} #{'✓' if r[:matched].any?}"
puts " GC paid / Dolibarr open: #{r[:gc_paid_dolibarr_open].size}#{r[:gc_paid_dolibarr_open].any? ? ' ← FIX with --fix' : ''}"
puts " Dolibarr paid / no GC: #{r[:dolibarr_paid_no_gc].size}#{r[:dolibarr_paid_no_gc].any? ? ' ← verify manually' : ''}"
puts " GC failed: #{r[:gc_failed].size}"
puts " GC cancelled: #{r[:gc_cancelled].size}"
puts " GC payment / no invoice: #{r[:gc_no_invoice].size}#{r[:gc_no_invoice].any? ? ' ← investigate' : ''}"
# --- Shine ↔ GoCardless Payouts ---
unless @result[:payout_matches].empty?
pm = r[:payout_matches]
verified = pm.count { |p| p.flag == Engine::PAYOUT_VERIFIED }
mismatch = pm.count { |p| p.flag == Engine::PAYOUT_AMOUNT_MISMATCH }
missing = pm.count { |p| p.flag == Engine::PAYOUT_MISSING }
expected = pm.sum(&:expected_amount_cents)
actual = pm.sum(&:actual_amount_cents)
diff = actual - expected
puts ""
puts "SHINE ↔ GOCARDLESS PAYOUTS"
puts " Payouts expected: #{pm.size}"
puts " Verified: #{verified}#{verified == pm.size ? ' ✓' : ''}"
puts " Amount mismatch: #{mismatch}#{mismatch > 0 ? ' ← check GC fees' : ''}"
puts " Missing in Shine: #{missing}#{missing > 0 ? ' ← investigate' : ''}"
puts " Expected total: #{format_eur(expected)}"
puts " Actual total: #{format_eur(actual)}"
puts " Difference: #{format_eur(diff)}#{diff.zero? ? ' ✓' : ' ← investigate'}"
end
# --- Action items ---
actions = r[:gc_paid_dolibarr_open] +
r[:dolibarr_paid_no_gc] +
r[:dolibarr_open_no_gc] +
r[:gc_no_invoice]
if actions.any?
puts ""
puts "ACTIONS NEEDED (#{actions.size})"
puts "-" * 60
r[:gc_paid_dolibarr_open].each_with_index do |m, i|
puts " #{i + 1}. [GC_PAID_DOLIBARR_OPEN] " \
"#{m.invoice.ref.ljust(16)} " \
"#{format_eur(m.invoice.amount_cents)} " \
"#{display_name(m.invoice).ljust(20)} " \
"GC: #{m.payment.id} #{m.payment.charge_date} (#{m.match_type})"
end
r[:dolibarr_paid_no_gc].each_with_index do |m, i|
n = r[:gc_paid_dolibarr_open].size + i + 1
puts " #{n}. [DOLIBARR_PAID_NO_GC] " \
"#{m.invoice.ref.ljust(16)} " \
"#{format_eur(m.invoice.amount_cents)} " \
"#{display_name(m.invoice).ljust(20)} " \
"No GoCardless payment found"
end
base = r[:gc_paid_dolibarr_open].size + r[:dolibarr_paid_no_gc].size
r[:dolibarr_open_no_gc].each_with_index do |inv, i|
n = base + i + 1
overdue = inv.due_date && inv.due_date < Date.today ? " (overdue since #{inv.due_date})" : ""
puts " #{n}. [DOLIBARR_OPEN_NO_GC] " \
"#{inv.ref.ljust(16)} " \
"#{format_eur(inv.amount_cents)} " \
"#{display_name(inv).ljust(20)} " \
"Open, no GC payment#{overdue}"
end
base2 = base + r[:dolibarr_open_no_gc].size
r[:gc_no_invoice].each_with_index do |m, i|
n = base2 + i + 1
puts " #{n}. [GC_NO_INVOICE] " \
"GC: #{m.payment.id} #{format_eur(m.payment.amount_cents)} " \
"\"#{m.payment.description}\" #{m.payment.customer_name} #{m.payment.charge_date}"
end
r[:payout_matches].select { |p| p.flag == Engine::PAYOUT_MISSING }.each do |pm|
n = base2 + r[:gc_no_invoice].size + 1
puts " #{n}. [PAYOUT_MISSING] " \
"Payout #{pm.payout_id} #{format_eur(pm.expected_amount_cents)} " \
"expected #{pm.payout_date}"
end
else
puts ""
puts " All clear — no actions needed."
end
puts ""
puts "=" * 60
csv_path = write_csv
puts " Report saved to: #{csv_path}"
puts ""
end
def write_csv
dir = "tmp"
Dir.mkdir(dir) unless Dir.exist?(dir)
path = "#{dir}/reconciliation_#{@to}.csv"
CSV.open(path, "w", encoding: "UTF-8") do |csv|
csv << %w[
invoice_ref customer_name amount_eur invoice_date due_date
dolibarr_status gc_payment_id gc_status gc_charge_date
match_type flag action
]
@result[:matches].each do |m|
inv = m.invoice
pay = m.payment
csv << [
inv&.ref,
inv&.customer_name,
inv ? "%.2f" % (inv.amount_cents / 100.0) : nil,
inv&.date,
inv&.due_date,
inv ? status_label(inv.status) : nil,
pay&.id,
pay&.status,
pay&.charge_date,
m.match_type,
m.flag,
action_label(m.flag)
]
end
@result[:dolibarr_open_no_gc].each do |inv|
csv << [
inv.ref, inv.customer_name,
"%.2f" % (inv.amount_cents / 100.0),
inv.date, inv.due_date,
status_label(inv.status),
nil, nil, nil, nil,
Engine::DOLIBARR_OPEN_NO_GC,
action_label(Engine::DOLIBARR_OPEN_NO_GC)
]
end
end
path
end
private
def display_name(inv)
name = inv.customer_name.to_s.strip
name.empty? ? "client_id:#{inv.customer_id}" : name
end
def format_eur(cents)
"€#{"%.2f" % (cents / 100.0)}"
end
def status_label(status)
{ 0 => "draft", 1 => "open", 2 => "paid", 3 => "cancelled" }[status] || status.to_s
end
def action_label(flag)
{
Engine::MATCHED => "none",
Engine::GC_PAID_DOLIBARR_OPEN => "mark_dolibarr_paid",
Engine::GC_FAILED => "verify",
Engine::GC_CANCELLED => "none",
Engine::GC_NO_INVOICE => "investigate",
Engine::DOLIBARR_PAID_NO_GC => "verify_manually",
Engine::DOLIBARR_OPEN_NO_GC => "follow_up"
}[flag] || flag.to_s
end
end
end

64
lib/reconciliation/shine_parser.rb Normal file
View File

@@ -0,0 +1,64 @@
# frozen_string_literal: true
require "csv"
require "date"
module Reconciliation
class ShineParser
Transaction = Struct.new(
:id,
:date,
:credit_cents, # Integer, > 0 for incoming money
:debit_cents, # Integer, > 0 for outgoing money
:label, # Libellé
:counterparty, # Nom de la contrepartie
keyword_init: true
)
def self.parse(csv_path)
content = File.read(csv_path, encoding: "UTF-8")
# Normalise Windows CRLF
content = content.gsub("\r\n", "\n").gsub("\r", "\n")
rows = CSV.parse(content, headers: true, col_sep: ";")
transactions = rows.filter_map do |row|
date = safe_parse_date(row["Date de la valeur"])
next unless date
Transaction.new(
id: row["Transaction ID"].to_s.strip,
date: date,
credit_cents: parse_french_amount(row["Crédit"]),
debit_cents: parse_french_amount(row["Débit"]),
label: row["Libellé"].to_s.strip,
counterparty: row["Nom de la contrepartie"].to_s.strip
)
end
$stderr.puts "[ShineParser] Loaded #{transactions.size} transactions from #{csv_path}"
transactions
end
# Returns all GoCardless credit transactions (incoming payouts from GC)
def self.gocardless_credits(transactions)
transactions.select do |t|
t.credit_cents > 0 &&
t.counterparty.upcase.include?("GOCARDLESS")
end
end
private_class_method def self.safe_parse_date(str)
return nil if str.nil? || str.strip.empty?
Date.strptime(str.strip, "%d/%m/%Y")
rescue Date::Error, ArgumentError
nil
end
# French decimal format: "51,10" → 5110 (cents)
private_class_method def self.parse_french_amount(str)
return 0 if str.nil? || str.strip.empty?
(str.strip.gsub(/\s/, "").gsub(",", ".").to_f * 100).round
end
end
end