📄 Viewing: MIGRATION_RUNBOOK.md
# MIGRATION RUNBOOK: Laravel SMS → Microservices
**FROM:** Laravel Monolith
**TO:** Go Messaging Service (phase 1), Spring Boot/Kotlin Dashboard (later phase)
**STRATEGY:** Strangler Pattern with Feature Flags (Incremental, Tenant-Level Rollout)
---
# PHASE A: SYSTEM BASELINE (SOURCE OF TRUTH)
## 0. HARD EXECUTION INVARIANTS (NON-NEGOTIABLE)
These invariants MUST be enforced at all times during migration.
### Invariant 1 — Single Execution Authority
At any moment, for a given `message_uuid`, exactly ONE system may perform the external provider send.
**Mode Behavior (Enforced):**
| Mode | Laravel Send | Go Send | Go dry_run | Quota Decrement | Fallback Allowed |
|--------|--------------|----------------|------------|-----------------|------------------|
| OFF | YES | NO | N/A | YES (legacy) | NO |
| SHADOW | YES | YES (dry_run) | true | NO | NO |
| ON | NO* | YES | false | YES | YES* |
*ON mode: Laravel fallback is allowed ONLY when Go request:
- times out (>500ms), OR
- returns transport failure, OR
- circuit breaker is OPEN
### Invariant 2 — UUID Generated ONCE at Entry Point
`$messageUuidMap` MUST be generated BEFORE any send logic executes.
UUID generation MUST NOT occur:
- Inside retry loops
- Inside provider loops
- Inside fallback paths
**Canonical Pattern:**
```php
$messageUuidMap = [];
foreach ($campaignSMSs as $sms) {
$messageUuidMap[$sms->id] = (string) \Str::uuid();
}
// All subsequent logic uses $messageUuidMap[$sms->id]
```
### Invariant 3 — Laravel is the ONLY Writer to sms_logs
Go service MUST NOT write to `sms_logs`.
Go writes ONLY to its internal `go_sms_logs` table.
Laravel writes Go results into `sms_logs` with:
- `source = 'go'`
- `message_uuid`
- `provider_message_id`
- `status`
### Invariant 4 — Quota Decrement Safety
Quota decrement MUST occur ONLY when:
- `mode == ON`
- AND `status IN ('accepted', 'duplicate')`
Quota MUST NOT decrement in SHADOW mode.
## 1. Evidence Summary (Repo-Backed)
**Entry Point (SMS Campaign Send):**
- `app/Http/Controllers/SmsController.php::campaignSendSms` (method definition).
Evidence: `app/Http/Controllers/SmsController.php` (search `public function campaignSendSms`).
**Tenant Resolution (Legacy):**
- `app/Models/Sms.php::scopeHasAgent` uses `agent_owner_id()` for Agents, otherwise `Auth::user()->id`.
Evidence: `app/Models/Sms.php::scopeHasAgent`, `app/Helpers.php::agent_owner_id`.
**Agent Owner Logic:**
- `app/Helpers.php::agent_owner_id` returns `Agent::where('user_id', Auth::id())->first()->assined_for_customer_id`.
Evidence: `app/Helpers.php::agent_owner_id`.
**Legacy Logging (smsLog):**
- `app/Helpers.php::smsLog` writes to `SmsLog` with `user_id`, `campaign_id`, `number`, `message_id`, `message`, `gateway`.
Evidence: `app/Helpers.php::smsLog`.
**Legacy Billing / Quota Decrement (SMS):**
- `EmailSMSLimitRate::where('owner_id', Auth::id())->decrement('sms', count(...))` appears in `campaignSendSms`.
Evidence: `app/Http/Controllers/SmsController.php` (search `EmailSMSLimitRate::where('owner_id', Auth::id())`).
**Providers Configured in Laravel UI & Send Logic (SMS):**
- Providers include: `twilio`, `nexmo`, `textlocal`, `plivo`, `signalwire`, `infobip`, `viber`, `whatsapp`, `telesign`, `sinch`, `clickatell`, `mailjet`, `lao`, `aakash`.
Evidence: `app/Http/Controllers/SmsController.php` switch cases containing these provider names.
**sms_logs Schema (Current):**
- `user_id`, `campaign_id`, `number`, `message_id`, `message`, `gateway`, timestamps.
Evidence: `database/migrations/2020_12_10_131409_create_sms_logs_table.php`.
**sms & sms_services Schemas (Credentials Storage):**
- Tables contain `sms_name`, `sms_id`, `sms_token`, `sms_from`, `sms_number`, `url`, `owner_id`.
Evidence: `database/migrations/2020_12_09_121240_create_sms_table.php`, `database/migrations/2022_04_28_131810_create_sms_services_table.php`.
**Session Driver Default:**
- `SESSION_DRIVER` defaults to `file`.
Evidence: `config/session.php` (`'driver' => env('SESSION_DRIVER', 'file')`).
**UNCONFIRMED (Requires Evidence):**
- Spring Boot/Kotlin Dashboard repo location, routes, or auth integration are not present in this repository.
Required evidence: repo URL or local path + controller/route definitions.
**UNCONFIRMED (Requires Evidence):**
- Direct PHP<->Java session deserialization feasibility.
Required evidence: storage format and serialization proof across stacks.
## 2. Canonical Contract (Defined Once; Referenced Everywhere)
**Endpoint:** `POST /internal/v1/sms/send-batch`
**Request:**
```json
{"tenant_id":123,"campaign_id":456,"messages":[{"message_uuid":"uuid","recipient":"+1234567890","body":"text","provider":"twilio"}],"dry_run":false}
```
**Response:**
```json
{"batch_id":"uuid","total":1,"accepted":1,"rejected":0,"results":[{"message_uuid":"uuid","status":"accepted","provider_message_id":"SM123","error_code":null,"error_message":null}]}
```
**Auth:** `X-Internal-API-Key` header
## 3. Canonical Tenant Resolution (REQUIRED)
**Function to add:** `resolveTenantId()`
```php
function resolveTenantId(): int {
if (!Auth::check()) {
if (App::runningInConsole()) {
$systemUserId = (int) config('app.system_user_id', 0);
if ($systemUserId > 0) {
return $systemUserId;
}
throw new RuntimeException('system_user_id not configured for console context');
}
throw new RuntimeException('Unauthenticated context: resolveTenantId requires Auth::check()');
}
if (Auth::user()->user_type === 'Agent') {
$ownerId = agent_owner_id();
if (!$ownerId) {
throw new RuntimeException('Agent owner id not found for authenticated Agent');
}
return $ownerId;
}
return Auth::id();
}
```
**Rule:** Replace all tenant-context `Auth::id()` lookups with `resolveTenantId()` for:
- Feature flags
- Quota decrement
- Go service credentials loading
- Internal API calls
---
# PHASE B: ATOMIC MIGRATION STEPS (Step 1..Step N)
## Step 1: Install Doctrine DBAL (Prerequisite)
### 1. Purpose & Pre-requisites
Enable `change()` operations in Laravel migrations.
### 2. Files Involved
- `composer.json`
- `composer.lock`
### 3. Commands to Run
```bash
composer require doctrine/dbal
```
### 4. Precise Code Changes
Composer dependency only.
### 5. Verification Gate
```bash
composer show doctrine/dbal
```
**STOP if FAIL. Execute rollback only.**
### 6. Rollback Path
```bash
composer remove doctrine/dbal
```
### 7. Artifacts Created/Changed
- `composer.json`
- `composer.lock`
### 8. Git Commit
```bash
git add composer.json composer.lock
git commit -m "chore: add doctrine/dbal prerequisite (Step 1)"
```
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 1 only: Install doctrine/dbal, verify it appears in composer show, commit if PASS.
```
---
## Step 2: Add Tracing Columns to sms_logs
### 1. Purpose & Pre-requisites
Add minimal tracing fields and `source` to `sms_logs`.
### 2. Files Involved
- `database/migrations/*_add_tracing_to_sms_logs.php`
### 3. Commands to Run
```bash
php artisan make:migration add_tracing_to_sms_logs
```
### 4. Precise Code Changes
```php
Schema::table('sms_logs', function (Blueprint $table) {
$table->char('message_uuid', 36)->nullable()->after('message_id');
$table->unsignedBigInteger('tenant_id')->nullable()->after('user_id');
$table->string('provider', 50)->nullable()->after('gateway');
$table->boolean('dry_run')->default(false)->after('provider');
$table->string('status', 20)->nullable()->after('dry_run');
$table->string('provider_message_id', 191)->nullable()->after('status');
$table->string('error_code', 50)->nullable()->after('provider_message_id');
$table->string('source', 20)->default('laravel')->after('error_code');
$table->index('message_uuid');
$table->index('tenant_id');
$table->index('source');
$table->index('provider');
$table->index(['source', 'created_at']);
});
```
### 5. Verification Gate
```bash
php artisan migrate
php artisan tinker --execute="
echo Schema::hasColumn('sms_logs','message_uuid') ? 'PASS' : 'FAIL';
"
php artisan tinker --execute="
\$driver = DB::getDriverName();
if (\$driver === 'mysql') {
\$rows = DB::select(\"SELECT INDEX_NAME FROM information_schema.statistics WHERE table_schema = DATABASE() AND table_name = 'sms_logs' AND index_name = 'sms_logs_source_created_at_index'\");
} elseif (\$driver === 'pgsql') {
\$rows = DB::select(\"SELECT indexname FROM pg_indexes WHERE tablename = 'sms_logs' AND indexname = 'sms_logs_source_created_at_index'\");
} else {
\$rows = [];
}
echo count(\$rows) > 0 ? 'PASS' : 'FAIL';
"
php artisan tinker --execute="
\$driver = DB::getDriverName();
if (\$driver === 'mysql') {
\$rows = DB::select(\"SELECT INDEX_NAME FROM information_schema.statistics WHERE table_schema = DATABASE() AND table_name = 'sms_logs' AND index_name = 'sms_logs_provider_index'\");
} elseif (\$driver === 'pgsql') {
\$rows = DB::select(\"SELECT indexname FROM pg_indexes WHERE tablename = 'sms_logs' AND indexname = 'sms_logs_provider_index'\");
} else {
\$rows = [];
}
echo count(\$rows) > 0 ? 'PASS' : 'FAIL';
"
```
**STOP if FAIL. Execute rollback only.**
### 6. Rollback Path
```bash
php artisan migrate:rollback --step=1
```
### 7. Artifacts Created/Changed
- `database/migrations/*_add_tracing_to_sms_logs.php`
### 8. Git Commit
```bash
git add database/migrations/*add_tracing_to_sms_logs*
git commit -m "feat: add tracing fields to sms_logs (Step 2)"
```
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 2: Add tracing fields migration for sms_logs. Run migrate, verify PASS, commit if PASS.
```
---
## Step 3: Add Canonical Tenant Resolver + Extend smsLog Signature
### 1. Purpose & Pre-requisites
Add `resolveTenantId()` and extend `smsLog()` to support tracing fields and source.
### 2. Files Involved
- `app/Helpers.php`
### 3. Commands to Run
```bash
cd .
```
### 4. Precise Code Changes
**Add after `agent_owner_id()` in `app/Helpers.php`:**
```php
/**
* Resolve tenant ID (canonical for migration)
*/
function resolveTenantId(): int
{
if (!Auth::check()) {
if (App::runningInConsole()) {
$systemUserId = (int) config('app.system_user_id', 0);
if ($systemUserId > 0) {
return $systemUserId;
}
throw new RuntimeException('system_user_id not configured for console context');
}
throw new RuntimeException('Unauthenticated context: resolveTenantId requires Auth::check()');
}
if (Auth::user()->user_type === 'Agent') {
$ownerId = agent_owner_id();
if (!$ownerId) {
throw new RuntimeException('Agent owner id not found for authenticated Agent');
}
return $ownerId;
}
return Auth::id();
}
```
**Replace `smsLog()` in `app/Helpers.php`:**
```php
function smsLog(
$campaign_id,
$number,
$message,
$gateway,
$message_uuid = null,
$source = 'laravel',
$tenant_id = null,
$provider = null,
$dry_run = false,
$status = null,
$provider_message_id = null,
$error_code = null
) {
$smsLog = new SmsLog();
$smsLog->user_id = Auth::user()->id;
$smsLog->campaign_id = $campaign_id;
$smsLog->number = $number;
$smsLog->message_id = Str::random(20);
$smsLog->message_uuid = $message_uuid ?? (string) Str::uuid();
$smsLog->message = $message;
$smsLog->gateway = $gateway;
$smsLog->source = $source;
$smsLog->tenant_id = $tenant_id ?? resolveTenantId();
$smsLog->provider = $provider ?? $gateway;
$smsLog->dry_run = $dry_run ? 1 : 0;
$smsLog->status = $status;
$smsLog->provider_message_id = $provider_message_id;
$smsLog->error_code = $error_code;
$smsLog->save();
return $smsLog;
}
```
**Usage Rules (Mandatory):**
- Laravel is the single writer to `sms_logs`.
- `ON` mode must call `smsLog(..., $source='go')`.
- `SHADOW` and `OFF` must use default `source='laravel'`.
- Go service must never write to `sms_logs`.
### 5. Verification Gate
```bash
php artisan tinker --execute="
\$log = smsLog(null, '+1234567890', 'Test', 'test');
if (empty(\$log->message_uuid)) exit(1);
echo 'PASS';
"
# Console path should return system_user_id
php artisan tinker --execute="echo resolveTenantId() === (int) config('app.system_user_id', 0) ? 'PASS' : 'FAIL';"
# Web unauthenticated should throw RuntimeException (simulate by clearing auth in a web request)
```
**STOP if FAIL. Execute rollback only.**
### 6. Rollback Path
```bash
git checkout HEAD -- app/Helpers.php
```
### 7. Artifacts Created/Changed
- `app/Helpers.php`
### 8. Git Commit
```bash
git add app/Helpers.php
git commit -m "feat: add resolveTenantId + extend smsLog (Step 3)"
```
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 3: Add resolveTenantId() and extend smsLog(). Verify tinker PASS, commit if PASS.
```
---
## Step 4: Backfill sms_logs Tracing Fields
### 1. Purpose & Pre-requisites
Backfill `message_uuid`, `tenant_id`, and `source` for existing rows.
### 2. Files Involved
- `app/Console/Commands/BackfillSmsLogTracing.php`
### 3. Commands to Run
```bash
php artisan make:command BackfillSmsLogTracing
```
### 4. Precise Code Changes
```php
<?php
namespace App\Console\Commands;
use Illuminate\Console\Command;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Log;
use Illuminate\Support\Str;
class BackfillSmsLogTracing extends Command
{
protected $signature = 'sms:backfill-tracing {--batch=1000}';
protected $description = 'Backfill sms_logs tracing fields';
public function handle()
{
$batch = (int) $this->option('batch');
$startedAt = now();
$this->info('BACKFILL_STARTED_AT=' . $startedAt->toDateTimeString());
DB::table('sms_logs')
->whereNull('message_uuid')
->orderBy('id')
->chunkById($batch, function ($rows) use ($startedAt) {
$batchStart = microtime(true);
DB::transaction(function () use ($rows) {
foreach ($rows as $row) {
DB::table('sms_logs')->where('id', $row->id)->update([
'message_uuid' => (string) Str::uuid(),
'source' => 'laravel',
'updated_at' => now(),
]);
}
});
$elapsedMs = (int) ((microtime(true) - $batchStart) * 1000);
$lastProcessedId = $rows->last()->id ?? null;
Log::info('sms:backfill-tracing uuid batch', [
'count' => count($rows),
'min_id' => $rows->first()->id ?? null,
'max_id' => $lastProcessedId,
'lastProcessedId' => $lastProcessedId,
'elapsed_ms' => $elapsedMs
]);
usleep(50000);
}, 'id');
// tenant_id backfill from users/agents
$minId = DB::table('sms_logs')->min('id');
$maxId = DB::table('sms_logs')->max('id');
if ($minId && $maxId) {
for ($start = $minId; $start <= $maxId; $start += $batch) {
$end = $start + $batch - 1;
$batchStart = microtime(true);
$processed = DB::affectingStatement("
UPDATE sms_logs l
SET tenant_id = CASE
WHEN (SELECT u.user_type FROM users u WHERE u.id = l.user_id) = 'Agent'
THEN (SELECT a.assined_for_customer_id FROM agents a WHERE a.user_id = l.user_id)
ELSE l.user_id
END
WHERE l.tenant_id IS NULL AND l.id BETWEEN ? AND ?
", [$start, $end]);
$elapsedMs = (int) ((microtime(true) - $batchStart) * 1000);
Log::info('sms:backfill-tracing tenant_id batch', [
'range' => [$start, $end],
'processed' => $processed,
'lastProcessedId' => $end,
'elapsed_ms' => $elapsedMs
]);
usleep(50000);
}
}
$remaining = DB::table('sms_logs')->whereNull('message_uuid')->count();
$this->info($remaining === 0 ? 'PASS' : "FAIL: {$remaining} NULL");
return $remaining === 0 ? 0 : 1;
}
}
```
### 5. Verification Gate
```bash
php artisan sms:backfill-tracing
# Capture BACKFILL_STARTED_AT from command output for rollback scope
php artisan tinker --execute="echo DB::table('sms_logs')->whereNull('message_uuid')->count() === 0 ? 'PASS' : 'FAIL';"
```
**STOP if FAIL. Execute rollback only.**
### 6. Rollback Path
```bash
php artisan tinker --execute="
\$startedAt = '<BACKFILL_STARTED_AT>';
DB::table('sms_logs')
->where('source', 'laravel')
->where('updated_at', '>=', \$startedAt)
->update(['message_uuid'=>null,'tenant_id'=>null]);
"
```
### 7. Artifacts Created/Changed
- `app/Console/Commands/BackfillSmsLogTracing.php`
### 8. Git Commit
```bash
git add app/Console/Commands/BackfillSmsLogTracing.php
git commit -m "feat: backfill sms_logs tracing (Step 4)"
```
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 4: Create BackfillSmsLogTracing command, run it, verify PASS, commit if PASS.
```
---
## Step 5: Enforce message_uuid NOT NULL + UNIQUE
### 1. Purpose & Pre-requisites
Ensure idempotency key is enforced.
### 2. Files Involved
- `database/migrations/*_enforce_message_uuid_unique.php`
### 3. Commands to Run
```bash
php artisan make:migration enforce_message_uuid_unique
```
### 4. Precise Code Changes
```php
Schema::table('sms_logs', function (Blueprint $table) {
$table->dropIndex(['message_uuid']);
$table->char('message_uuid', 36)->nullable(false)->change();
$table->unique('message_uuid');
});
```
**Down migration:**
```php
Schema::table('sms_logs', function (Blueprint $table) {
$table->dropUnique(['message_uuid']);
$table->char('message_uuid', 36)->nullable()->change();
$table->index('message_uuid');
});
```
### 5. Verification Gate
```bash
php artisan tinker --execute="
\$nulls = DB::table('sms_logs')->whereNull('message_uuid')->count();
echo \$nulls === 0 ? 'PASS' : 'FAIL';
"
php artisan migrate
php artisan tinker --execute="
try {
DB::table('sms_logs')->insert([
'user_id'=>1,'tenant_id'=>1,'message_uuid'=>DB::table('sms_logs')->first()->message_uuid,
'number'=>'x','message_id'=>'x','message'=>'x','gateway'=>'x','created_at'=>now(),'updated_at'=>now()
]);
echo 'FAIL';
} catch (Exception \$e) {
echo 'PASS';
}
"
```
**STOP if FAIL. Execute rollback only.**
### 6. Rollback Path
```bash
php artisan migrate:rollback --step=1
```
### 7. Artifacts Created/Changed
- `database/migrations/*_enforce_message_uuid_unique.php`
### 8. Git Commit
```bash
git add database/migrations/*enforce_message_uuid_unique*
git commit -m "feat: enforce message_uuid unique (Step 5)"
```
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 5: Create message_uuid unique migration, run migrate, verify PASS, commit if PASS.
```
---
## Step 6: Create Feature Flags Table
### 1. Purpose & Pre-requisites
Tenant-level feature flag control for gradual rollout.
### 2. Files Involved
- `database/migrations/*_create_feature_flags_table.php`
- `app/Models/FeatureFlag.php`
### 3. Commands to Run
```bash
php artisan make:migration create_feature_flags_table
php artisan make:model FeatureFlag
```
### 4. Precise Code Changes
**Migration:**
```php
Schema::create('feature_flags', function (Blueprint $table) {
$table->id();
$table->unsignedBigInteger('tenant_id')->nullable();
$table->string('feature_name', 100);
$table->enum('mode', ['OFF', 'SHADOW', 'ON'])->default('OFF');
$table->timestamps();
$table->unique(['tenant_id', 'feature_name']);
});
DB::table('feature_flags')->insert([
'tenant_id' => null,
'feature_name' => 'messaging_service',
'mode' => 'OFF',
'created_at' => now(),
'updated_at' => now()
]);
```
**Model:**
```php
<?php
namespace App\Models;
use Illuminate\Database\Eloquent\Model;
use Illuminate\Support\Facades\Cache;
class FeatureFlag extends Model
{
protected $fillable = ['tenant_id', 'feature_name', 'mode'];
public static function getMode(string $feature, ?int $tenantId = null): string
{
$tenantId = $tenantId ?? resolveTenantId();
$cacheKey = "feature_flag:{$feature}:tenant:" . ($tenantId ?? 'global');
return Cache::remember($cacheKey, 60, function () use ($feature, $tenantId) {
$flag = self::where('tenant_id', $tenantId)->where('feature_name', $feature)->first();
if ($flag) return $flag->mode;
$global = self::whereNull('tenant_id')->where('feature_name', $feature)->first();
return $global ? $global->mode : 'OFF';
});
}
}
```
### 5. Verification Gate
```bash
php artisan migrate
php artisan tinker --execute="echo App\Models\FeatureFlag::getMode('messaging_service') === 'OFF' ? 'PASS' : 'FAIL';"
```
**STOP if FAIL. Execute rollback only.**
### 6. Rollback Path
```bash
php artisan migrate:rollback --step=1
rm app/Models/FeatureFlag.php
```
### 7. Artifacts Created/Changed
- `database/migrations/*_create_feature_flags_table.php`
- `app/Models/FeatureFlag.php`
### 8. Git Commit
```bash
git add database/migrations/*feature_flags* app/Models/FeatureFlag.php
git commit -m "feat: add feature_flags (Step 6)"
```
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 6: Create feature_flags table + model, run migrate, verify PASS, commit if PASS.
```
---
## Step 7: Add Internal API Key + Service URL
### 1. Purpose & Pre-requisites
Secure internal calls to Go service.
### 2. Files Involved
- `.env`
- `.env.example`
### 3. Commands to Run
```bash
php artisan tinker --execute="echo base64_encode(random_bytes(32));"
```
### 4. Precise Code Changes
Add to `.env`:
```
INTERNAL_API_KEY=<generated>
MESSAGING_SERVICE_URL=http://localhost:8080
```
Add to `.env.example`:
```
INTERNAL_API_KEY=
MESSAGING_SERVICE_URL=
```
### 5. Verification Gate
```bash
php artisan tinker --execute="echo strlen(env('INTERNAL_API_KEY'))>=32 ? 'PASS' : 'FAIL';"
```
**STOP if FAIL. Execute rollback only.**
### 6. Rollback Path
Remove the lines from `.env` and `.env.example`.
### 7. Artifacts Created/Changed
- `.env`
- `.env.example`
### 8. Git Commit
```bash
git add .env.example
git commit -m "feat: add internal api key config (Step 7)"
```
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 7: Add INTERNAL_API_KEY and MESSAGING_SERVICE_URL to envs. Commit .env.example only.
```
---
## Step 8: MessagingServiceClient + 500ms Budget
### 1. Purpose & Pre-requisites
Provide internal client with 500ms timeout budget for circuit breaker decisions.
### 2. Files Involved
- `app/Services/MessagingServiceClient.php`
- `config/services.php`
### 3. Commands to Run
```bash
mkdir -p app/Services
```
### 4. Precise Code Changes
**config/services.php (add):**
```php
'messaging' => [
'url' => env('MESSAGING_SERVICE_URL', 'http://localhost:8080'),
],
```
**app/Services/MessagingServiceClient.php:**
```php
<?php
namespace App\Services;
use Illuminate\Support\Facades\Http;
class MessagingServiceClient
{
private string $baseUrl;
private string $apiKey;
public function __construct()
{
$this->baseUrl = config('services.messaging.url');
$this->apiKey = env('INTERNAL_API_KEY', '');
}
public function sendBatch(int $tenantId, array $messages, ?int $campaignId = null, bool $dryRun = false): array
{
// INVARIANT 2: UUID must be pre-generated by caller. Reject if missing.
foreach ($messages as $msg) {
if (empty($msg['message_uuid'])) {
throw new \InvalidArgumentException('message_uuid is required for each message (Invariant 2)');
}
}
$response = Http::timeout(0.5)
->withHeaders(['X-Internal-API-Key' => $this->apiKey])
->post($this->baseUrl . '/internal/v1/sms/send-batch', [
'tenant_id' => $tenantId,
'campaign_id' => $campaignId,
'messages' => $messages,
'dry_run' => $dryRun
]);
if (!$response->successful()) {
throw new \Exception("Messaging service error: " . $response->status());
}
return $response->json();
}
public function isHealthy(): bool
{
try {
return Http::timeout(0.5)->get($this->baseUrl . '/health')->successful();
} catch (\Exception $e) {
return false;
}
}
}
```
### 5. Verification Gate
```bash
php artisan tinker --execute="new App\Services\MessagingServiceClient(); echo 'PASS';"
```
**STOP if FAIL. Execute rollback only.**
### 6. Rollback Path
```bash
rm app/Services/MessagingServiceClient.php
git checkout HEAD -- config/services.php
```
### 7. Artifacts Created/Changed
- `app/Services/MessagingServiceClient.php`
- `config/services.php`
### 8. Git Commit
```bash
git add app/Services/MessagingServiceClient.php config/services.php
git commit -m "feat: add MessagingServiceClient (Step 8)"
```
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 8: Add MessagingServiceClient with 500ms timeout, verify tinker PASS, commit if PASS.
```
---
## Step 9: Circuit Breaker + Fallback Verification Command
### 1. Purpose & Pre-requisites
Provide resilient fallback when Go fails or exceeds 500ms, and add an executable fallback verification command.
### 2. Files Involved
- `app/Services/CircuitBreaker.php`
- `app/Console/Commands/VerifyGoFallback.php`
### 3. Commands to Run
```bash
php artisan make:command VerifyGoFallback
```
### 4. Precise Code Changes
**app/Services/CircuitBreaker.php:**
```php
<?php
namespace App\Services;
use Illuminate\Support\Facades\Cache;
class CircuitBreaker
{
private string $service;
private int $threshold = 5;
private int $timeout = 60;
private int $halfOpenAttempts = 1;
public function __construct(string $service)
{
$this->service = $service;
}
public function isOpen(): bool
{
return Cache::get("circuit:{$this->service}:open", false);
}
public function allowRequest(): bool
{
if (!$this->isOpen()) {
return true;
}
$attempts = (int) Cache::get("circuit:{$this->service}:half_open_attempts", 0);
if ($attempts < $this->halfOpenAttempts) {
Cache::put("circuit:{$this->service}:half_open_attempts", $attempts + 1, $this->timeout);
return true;
}
return false;
}
public function recordFailure(): void
{
$failures = Cache::increment("circuit:{$this->service}:failures");
if ($failures >= $this->threshold) {
Cache::put("circuit:{$this->service}:open", true, $this->timeout);
Cache::forget("circuit:{$this->service}:failures");
Cache::forget("circuit:{$this->service}:half_open_attempts");
}
}
public function recordSuccess(): void
{
Cache::forget("circuit:{$this->service}:failures");
Cache::forget("circuit:{$this->service}:open");
Cache::forget("circuit:{$this->service}:half_open_attempts");
}
}
```
**app/Console/Commands/VerifyGoFallback.php:**
```php
<?php
namespace App\Console\Commands;
use Illuminate\Console\Command;
use Illuminate\Support\Facades\Http;
use Illuminate\Support\Str;
use App\Services\CircuitBreaker;
use App\Services\MessagingServiceClient;
class VerifyGoFallback extends Command
{
protected $signature = 'sms:verify-fallback';
protected $description = 'Verify circuit breaker opens and fallback is used on Go failures';
public function handle()
{
Http::fake([
'*' => Http::response(['error' => 'fail'], 500),
]);
$cb = new CircuitBreaker('messaging_service');
$client = new MessagingServiceClient();
$tenantId = 1;
$messages = [[
'message_uuid' => (string) Str::uuid(),
'recipient' => '+10000000000',
'body' => 'test',
'provider' => 'twilio'
]];
for ($i = 0; $i < 5; $i++) {
try {
$client->sendBatch($tenantId, $messages, null, true);
} catch (\Exception $e) {
$cb->recordFailure();
}
}
if (!$cb->isOpen()) {
$this->error('FAIL: circuit did not open');
return 1;
}
$this->info('PASS: circuit open, fallback must be used in controller');
return 0;
}
}
```
### 5. Verification Gate
```bash
php artisan sms:verify-fallback
```
**STOP if FAIL. Execute rollback only.**
### 6. Rollback Path
```bash
rm app/Services/CircuitBreaker.php app/Console/Commands/VerifyGoFallback.php
```
### 7. Artifacts Created/Changed
- `app/Services/CircuitBreaker.php`
- `app/Console/Commands/VerifyGoFallback.php`
### 8. Git Commit
```bash
git add app/Services/CircuitBreaker.php app/Console/Commands/VerifyGoFallback.php
git commit -m "feat: add circuit breaker + fallback verification (Step 9)"
```
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 9: Add CircuitBreaker + VerifyGoFallback command, run verification, commit if PASS.
```
---
## Step 10: Create go_sms_logs Table (Go Idempotency Store)
### 1. Purpose & Pre-requisites
Create Go-side log table for idempotency and reconciliation.
### 2. Files Involved
- `database/migrations/*_create_go_sms_logs_table.php`
### 3. Commands to Run
```bash
php artisan make:migration create_go_sms_logs_table
```
### 4. Precise Code Changes
```php
Schema::create('go_sms_logs', function (Blueprint $table) {
$table->id();
$table->char('message_uuid', 36)->unique();
$table->unsignedBigInteger('tenant_id');
$table->string('provider', 50);
$table->boolean('dry_run')->default(false);
$table->string('status', 20);
$table->string('provider_message_id', 191)->nullable();
$table->string('error_code', 50)->nullable();
$table->longText('error_message')->nullable();
$table->timestamps();
$table->index('tenant_id');
$table->index(['status', 'created_at']);
});
```
### 5. Verification Gate
```bash
php artisan migrate
php artisan tinker --execute="
echo Schema::hasColumn('go_sms_logs','message_uuid') ? 'PASS' : 'FAIL';
"
php artisan tinker --execute="
\$driver = DB::getDriverName();
if (\$driver === 'mysql') {
\$rows = DB::select(\"SELECT INDEX_NAME FROM information_schema.statistics WHERE table_schema = DATABASE() AND table_name = 'go_sms_logs' AND index_name = 'go_sms_logs_status_created_at_index'\");
} elseif (\$driver === 'pgsql') {
\$rows = DB::select(\"SELECT indexname FROM pg_indexes WHERE tablename = 'go_sms_logs' AND indexname = 'go_sms_logs_status_created_at_index'\");
} else {
\$rows = [];
}
echo count(\$rows) > 0 ? 'PASS' : 'FAIL';
"
```
**STOP if FAIL. Execute rollback only.**
### 6. Rollback Path
```bash
php artisan migrate:rollback --step=1
```
### 7. Artifacts Created/Changed
- `database/migrations/*_create_go_sms_logs_table.php`
### 8. Git Commit
```bash
git add database/migrations/*go_sms_logs*
git commit -m "feat: add go_sms_logs table (Step 10)"
```
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 10: Create go_sms_logs migration, run migrate, verify PASS, commit if PASS.
```
---
## Step 11: Create Go Messaging Service Skeleton
### 1. Purpose & Pre-requisites
Create Go service with auth middleware and health endpoint.
### 2. Files Involved
- `services/messaging-service/go.mod`
- `services/messaging-service/cmd/server/main.go`
- `services/messaging-service/internal/middleware/auth.go`
- `services/messaging-service/internal/handlers/health.go`
- `services/messaging-service/internal/handlers/sms.go`
- `services/messaging-service/internal/models/models.go`
- `services/messaging-service/Makefile`
### 3. Commands to Run
```bash
mkdir -p services/messaging-service/cmd/server
mkdir -p services/messaging-service/internal/{middleware,handlers,models,providers,ratelimit,worker}
cd services/messaging-service
go mod init messaging-service
go get github.com/gin-gonic/gin@v1.9.1
go get gorm.io/gorm@v1.25.5
go get gorm.io/driver/mysql@v1.5.2
```
### 4. Precise Code Changes
**services/messaging-service/cmd/server/main.go:**
```go
package main
import (
"log"
"os"
"messaging-service/internal/handlers"
"messaging-service/internal/middleware"
"github.com/gin-gonic/gin"
)
func main() {
port := os.Getenv("PORT")
if port == "" {
port = "8080"
}
r := gin.Default()
r.GET("/health", handlers.Health)
internal := r.Group("/internal/v1")
internal.Use(middleware.APIKeyAuth())
internal.POST("/sms/send-batch", handlers.SendBatch)
log.Printf("Starting on :%s", port)
r.Run(":" + port)
}
```
**services/messaging-service/internal/middleware/auth.go:**
```go
package middleware
import (
"net/http"
"os"
"github.com/gin-gonic/gin"
)
func APIKeyAuth() gin.HandlerFunc {
return func(c *gin.Context) {
key := c.GetHeader("X-Internal-API-Key")
expected := os.Getenv("INTERNAL_API_KEY")
if expected == "" || key != expected {
c.AbortWithStatusJSON(http.StatusUnauthorized, gin.H{"error": "unauthorized"})
return
}
c.Next()
}
}
```
**services/messaging-service/internal/handlers/health.go:**
```go
package handlers
import (
"net/http"
"github.com/gin-gonic/gin"
)
func Health(c *gin.Context) {
c.JSON(http.StatusOK, gin.H{"status": "healthy"})
}
```
**services/messaging-service/internal/models/models.go:**
```go
package models
import "time"
type SmsCredential struct {
ID uint `gorm:"primaryKey"`
OwnerID uint `gorm:"column:owner_id"`
SmsName string `gorm:"column:sms_name"`
SmsID string `gorm:"column:sms_id"`
SmsToken string `gorm:"column:sms_token"`
SmsFrom string `gorm:"column:sms_from"`
SmsNumber string `gorm:"column:sms_number"`
URL string `gorm:"column:url"`
}
func (SmsCredential) TableName() string { return "sms" }
type SmsService struct {
ID uint `gorm:"primaryKey"`
OwnerID uint `gorm:"column:owner_id"`
SmsName string `gorm:"column:sms_name"`
SmsID string `gorm:"column:sms_id"`
SmsToken string `gorm:"column:sms_token"`
SmsFrom string `gorm:"column:sms_from"`
SmsNumber string `gorm:"column:sms_number"`
URL string `gorm:"column:url"`
Status bool `gorm:"column:status"`
}
func (SmsService) TableName() string { return "sms_services" }
type GoSmsLog struct {
ID uint `gorm:"primaryKey"`
MessageUUID string `gorm:"column:message_uuid"`
TenantID uint `gorm:"column:tenant_id"`
Provider string `gorm:"column:provider"`
DryRun bool `gorm:"column:dry_run"`
Status string `gorm:"column:status"`
ProviderMessageID string `gorm:"column:provider_message_id"`
ErrorCode string `gorm:"column:error_code"`
ErrorMessage string `gorm:"column:error_message"`
CreatedAt time.Time `gorm:"column:created_at"`
UpdatedAt time.Time `gorm:"column:updated_at"`
}
func (GoSmsLog) TableName() string { return "go_sms_logs" }
```
**services/messaging-service/Makefile:**
```makefile
run:
: "$${DB_DSN?Set DB_DSN}"
INTERNAL_API_KEY=test PORT=8080 DB_DSN=$$DB_DSN go run cmd/server/main.go
build:
go build -o bin/server cmd/server/main.go
```
### 5. Verification Gate
```bash
cd services/messaging-service
go build -o /dev/null cmd/server/main.go && echo "Build PASS"
```
**STOP if FAIL. Execute rollback only.**
### 6. Rollback Path
```bash
rm -rf services/messaging-service
```
### 7. Artifacts Created/Changed
- `services/messaging-service/` directory
### 8. Git Commit
```bash
git add services/messaging-service/
git commit -m "feat: init go messaging service skeleton (Step 11)"
```
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 11: Create Go service skeleton and build; commit if PASS.
```
---
## Step 12: Harden Go DB Lifecycle Determinism
### 1. Purpose & Pre-requisites
Ensure deterministic startup: if DB connection fails, service exits immediately with fatal log.
### 2. Files Involved
- `services/messaging-service/internal/handlers/sms.go`
- `services/messaging-service/internal/handlers/retry_test.go`
### 3. Commands to Run
N/A
### 4. Precise Code Changes
**In `services/messaging-service/internal/handlers/sms.go` init():**
```go
var db *gorm.DB
func init() {
dsn := os.Getenv("DB_DSN")
if dsn == "" {
log.Fatal("DB_DSN is required")
}
var err error
db, err = gorm.Open(mysql.Open(dsn), &gorm.Config{})
if err != nil {
log.Fatal("DB connection failed: ", err)
}
}
```
### 5. Verification Gate
```bash
cd services/messaging-service
DB_DSN=invalid INTERNAL_API_KEY=test PORT=8080 go run cmd/server/main.go
```
**Expected:** process exits immediately with fatal log.
**STOP if process keeps running. Execute rollback only.**
### 6. Rollback Path
```bash
git checkout HEAD -- services/messaging-service/internal/handlers/sms.go
rm services/messaging-service/internal/handlers/retry_test.go
```
### 7. Artifacts Created/Changed
- `services/messaging-service/internal/handlers/sms.go`
- `services/messaging-service/internal/handlers/retry_test.go`
### 8. Git Commit
```bash
git add services/messaging-service/internal/handlers/sms.go
git commit -m "fix: enforce fatal DB connection failure (Step 12)"
```
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 12: Harden Go DB init to fail fast; verify invalid DB_DSN exits; commit if PASS.
```
---
## Step 13: Add Shared HTTP Client (Provider Timeout)
### 1. Purpose & Pre-requisites
Ensure provider requests have deterministic timeouts.
### 2. Files Involved
- `services/messaging-service/internal/providers/http_client.go`
### 3. Commands to Run
N/A
### 4. Precise Code Changes
```go
package providers
import (
"net/http"
"time"
)
var httpClient = &http.Client{
Timeout: 5 * time.Second,
}
```
### 5. Verification Gate
```bash
cd services/messaging-service
go test ./...
```
**STOP if FAIL. Execute rollback only.**
### 6. Rollback Path
```bash
rm services/messaging-service/internal/providers/http_client.go
```
### 7. Artifacts Created/Changed
- `services/messaging-service/internal/providers/http_client.go`
### 8. Git Commit
```bash
git add services/messaging-service/internal/providers/http_client.go
git commit -m "feat: add provider http client timeout (Step 13)"
```
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 13: Add shared http client timeout; run go test; commit if PASS.
```
---
## Step 14: Implement Worker Pool + Per-Tenant Rate Limiter + Tests
### 1. Purpose & Pre-requisites
Provide fixed worker pool and per-tenant token bucket limiter, with unit tests.
### 2. Files Involved
- `services/messaging-service/internal/worker/pool.go`
- `services/messaging-service/internal/worker/pool_test.go`
- `services/messaging-service/internal/ratelimit/tenant_limiter.go`
- `services/messaging-service/internal/ratelimit/tenant_limiter_test.go`
### 3. Commands to Run
N/A
### 4. Precise Code Changes
**services/messaging-service/internal/worker/pool.go:**
```go
package worker
import (
"os"
"strconv"
"sync"
)
type Pool struct {
wg sync.WaitGroup
jobs chan func()
}
func NewPool() *Pool {
workers := 20
if v := os.Getenv("WORKERS"); v != "" {
if n, err := strconv.Atoi(v); err == nil && n > 0 {
workers = n
}
}
p := &Pool{
jobs: make(chan func()),
}
for i := 0; i < workers; i++ {
go func() {
for job := range p.jobs {
job()
p.wg.Done()
}
}()
}
return p
}
func (p *Pool) Submit(job func()) {
p.wg.Add(1)
p.jobs <- job
}
func (p *Pool) Wait() {
p.wg.Wait()
}
```
**services/messaging-service/internal/worker/pool_test.go:**
```go
package worker
import (
"sync/atomic"
"testing"
)
func TestPoolExecutesJobs(t *testing.T) {
p := NewPool()
var count int32
for i := 0; i < 10; i++ {
p.Submit(func() {
atomic.AddInt32(&count, 1)
})
}
p.Wait()
if count != 10 {
t.Fatalf("expected 10, got %d", count)
}
}
```
**services/messaging-service/internal/ratelimit/tenant_limiter.go:**
```go
package ratelimit
import (
"os"
"strconv"
"sync"
"time"
)
type tokenBucket struct {
capacity int
tokens float64
last time.Time
rate float64
}
type TenantLimiter struct {
mu sync.Mutex
buckets map[uint]*tokenBucket
rate float64
burst int
}
func NewTenantLimiter() *TenantLimiter {
rate := 10.0
if v := os.Getenv("TENANT_RPS"); v != "" {
if n, err := strconv.Atoi(v); err == nil && n > 0 {
rate = float64(n)
}
}
burst := 20
if v := os.Getenv("TENANT_BURST"); v != "" {
if n, err := strconv.Atoi(v); err == nil && n > 0 {
burst = n
}
}
return &TenantLimiter{
buckets: map[uint]*tokenBucket{},
rate: rate,
burst: burst,
}
}
func (l *TenantLimiter) Allow(tenantID uint) bool {
l.mu.Lock()
defer l.mu.Unlock()
b, ok := l.buckets[tenantID]
if !ok {
b = &tokenBucket{capacity: l.burst, tokens: float64(l.burst), last: time.Now(), rate: l.rate}
l.buckets[tenantID] = b
}
now := time.Now()
elapsed := now.Sub(b.last).Seconds()
b.tokens = min(float64(b.capacity), b.tokens+elapsed*b.rate)
b.last = now
if b.tokens >= 1 {
b.tokens -= 1
return true
}
return false
}
func min(a, b float64) float64 {
if a < b {
return a
}
return b
}
```
**services/messaging-service/internal/ratelimit/tenant_limiter_test.go:**
```go
package ratelimit
import "testing"
func TestLimiterIsPerTenant(t *testing.T) {
l := NewTenantLimiter()
// Exhaust tenant 1
for i := 0; i < 100; i++ {
l.Allow(1)
}
// Tenant 2 should still be allowed initially
if !l.Allow(2) {
t.Fatalf("tenant 2 should not be starved by tenant 1")
}
}
```
### 5. Verification Gate
```bash
cd services/messaging-service
go test ./...
```
**STOP if FAIL. Execute rollback only.**
### 6. Rollback Path
```bash
rm services/messaging-service/internal/worker/pool.go services/messaging-service/internal/worker/pool_test.go
rm services/messaging-service/internal/ratelimit/tenant_limiter.go services/messaging-service/internal/ratelimit/tenant_limiter_test.go
```
### 7. Artifacts Created/Changed
- `services/messaging-service/internal/worker/pool.go`
- `services/messaging-service/internal/worker/pool_test.go`
- `services/messaging-service/internal/ratelimit/tenant_limiter.go`
- `services/messaging-service/internal/ratelimit/tenant_limiter_test.go`
### 8. Git Commit
```bash
git add services/messaging-service/internal/worker/pool.go services/messaging-service/internal/worker/pool_test.go
git add services/messaging-service/internal/ratelimit/tenant_limiter.go services/messaging-service/internal/ratelimit/tenant_limiter_test.go
git commit -m "feat: worker pool + tenant limiter + tests (Step 14)"
```
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 14: Add worker pool + rate limiter + tests; run go test; commit if PASS.
```
---
## Step 15: Implement Go Idempotency + Logging (go_sms_logs)
### 1. Purpose & Pre-requisites
Prevent duplicate sends by `message_uuid`, log deterministic responses, and enforce dry_run behavior.
### 2. Files Involved
- `services/messaging-service/internal/handlers/sms.go`
### 3. Commands to Run
N/A
### 4. Precise Code Changes
**Replace `services/messaging-service/internal/handlers/sms.go` with:**
```go
package handlers
import (
"errors"
"log"
"math/rand"
"net"
"net/http"
"os"
"strings"
"sync"
"time"
"github.com/gin-gonic/gin"
"gorm.io/driver/mysql"
"gorm.io/gorm"
"messaging-service/internal/models"
"messaging-service/internal/providers"
"messaging-service/internal/ratelimit"
"messaging-service/internal/worker"
)
var db *gorm.DB
func init() {
dsn := os.Getenv("DB_DSN")
if dsn == "" {
log.Fatal("DB_DSN is required")
}
var err error
db, err = gorm.Open(mysql.Open(dsn), &gorm.Config{})
if err != nil {
log.Fatal("DB connection failed: ", err)
}
}
type Message struct {
MessageUUID string `json:"message_uuid" binding:"required"`
Recipient string `json:"recipient" binding:"required"`
Body string `json:"body" binding:"required"`
Provider string `json:"provider" binding:"required"`
}
type SendBatchRequest struct {
TenantID int `json:"tenant_id" binding:"required"`
CampaignID *int `json:"campaign_id"`
Messages []Message `json:"messages" binding:"required"`
DryRun bool `json:"dry_run"`
}
type SendResult struct {
MessageUUID string `json:"message_uuid"`
Status string `json:"status"`
ProviderMessageID string `json:"provider_message_id"`
ErrorCode string `json:"error_code"`
ErrorMessage string `json:"error_message"`
}
func loadCredentials(tenantID int, provider string) (models.SmsCredential, error) {
var svc models.SmsService
if err := db.Where("owner_id = ? AND sms_name = ? AND status = ?", tenantID, provider, true).First(&svc).Error; err == nil {
return models.SmsCredential{
OwnerID: svc.OwnerID,
SmsName: svc.SmsName,
SmsID: svc.SmsID,
SmsToken: svc.SmsToken,
SmsFrom: svc.SmsFrom,
SmsNumber: svc.SmsNumber,
URL: svc.URL,
}, nil
}
var cred models.SmsCredential
if err := db.Where("owner_id = ? AND sms_name = ?", tenantID, provider).First(&cred).Error; err != nil {
if errors.Is(err, gorm.ErrRecordNotFound) {
return models.SmsCredential{}, errors.New("credentials not found for provider: " + provider)
}
return models.SmsCredential{}, err
}
return cred, nil
}
func loadExistingLog(messageUUID string) (*models.GoSmsLog, error) {
var logEntry models.GoSmsLog
if err := db.Where("message_uuid = ?", messageUUID).First(&logEntry).Error; err != nil {
return nil, err
}
return &logEntry, nil
}
func createLog(entry *models.GoSmsLog) error {
return db.Create(entry).Error
}
func updateLog(entry *models.GoSmsLog, status, providerMessageID, errorCode, errorMessage string) {
db.Model(entry).Updates(map[string]interface{}{
"status": status,
"provider_message_id": providerMessageID,
"error_code": errorCode,
"error_message": errorMessage,
"updated_at": time.Now(),
})
}
func SendBatch(c *gin.Context) {
var req SendBatchRequest
if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
workers := worker.NewPool()
limiter := ratelimit.NewTenantLimiter()
var resultsMu sync.Mutex
results := make([]SendResult, len(req.Messages))
for i, msg := range req.Messages {
if existing, err := loadExistingLog(msg.MessageUUID); err == nil {
resultsMu.Lock()
results[i] = SendResult{
MessageUUID: existing.MessageUUID,
Status: "duplicate",
ProviderMessageID: existing.ProviderMessageID,
ErrorCode: existing.ErrorCode,
ErrorMessage: existing.ErrorMessage,
}
resultsMu.Unlock()
continue
}
if !limiter.Allow(uint(req.TenantID)) {
resultsMu.Lock()
results[i] = SendResult{
MessageUUID: msg.MessageUUID,
Status: "rejected",
ErrorCode: "RATE_LIMIT",
ErrorMessage: "tenant rate limit exceeded",
}
resultsMu.Unlock()
continue
}
entry := &models.GoSmsLog{
MessageUUID: msg.MessageUUID,
TenantID: uint(req.TenantID),
Provider: msg.Provider,
DryRun: req.DryRun,
Status: "queued",
}
if err := createLog(entry); err != nil {
resultsMu.Lock()
results[i] = SendResult{
MessageUUID: msg.MessageUUID,
Status: "rejected",
ErrorCode: "LOG_CREATE_FAILED",
ErrorMessage: err.Error(),
}
resultsMu.Unlock()
continue
}
index := i
message := msg
workers.Submit(func() {
if req.DryRun {
log.Printf("DRY_RUN_NO_PROVIDER_CALL: message_uuid=%s", message.MessageUUID)
updateLog(entry, "accepted", "", "", "")
resultsMu.Lock()
results[index] = SendResult{
MessageUUID: message.MessageUUID,
Status: "accepted",
}
resultsMu.Unlock()
return
}
cred, err := loadCredentials(req.TenantID, message.Provider)
if err != nil {
updateLog(entry, "rejected", "", "CREDENTIALS_NOT_FOUND", err.Error())
resultsMu.Lock()
results[index] = SendResult{
MessageUUID: message.MessageUUID,
Status: "rejected",
ErrorCode: "CREDENTIALS_NOT_FOUND",
ErrorMessage: err.Error(),
}
resultsMu.Unlock()
return
}
providerMsgID, err := retrySend(func() (string, error) {
return providers.Send(message.Provider, cred.SmsID, cred.SmsToken, cred.SmsFrom, cred.URL, message.Recipient, message.Body)
})
if err != nil {
updateLog(entry, "rejected", "", "PROVIDER_ERROR", err.Error())
resultsMu.Lock()
results[index] = SendResult{
MessageUUID: message.MessageUUID,
Status: "rejected",
ErrorCode: "PROVIDER_ERROR",
ErrorMessage: err.Error(),
}
resultsMu.Unlock()
return
}
updateLog(entry, "accepted", providerMsgID, "", "")
resultsMu.Lock()
results[index] = SendResult{
MessageUUID: message.MessageUUID,
Status: "accepted",
ProviderMessageID: providerMsgID,
}
resultsMu.Unlock()
})
}
workers.Wait()
accepted := 0
for _, r := range results {
if r.Status == "accepted" || r.Status == "duplicate" {
accepted++
}
}
c.JSON(http.StatusOK, gin.H{
"batch_id": "batch-" + req.Messages[0].MessageUUID[:8],
"total": len(req.Messages),
"accepted": accepted,
"rejected": len(req.Messages) - accepted,
"results": results,
})
}
func retrySend(sendFn func() (string, error)) (string, error) {
const maxAttempts = 3
const maxTotal = 2 * time.Second
start := time.Now()
var lastErr error
for attempt := 1; attempt <= maxAttempts; attempt++ {
msgID, err := sendFn()
if err == nil {
return msgID, nil
}
lastErr = err
if !isTransient(err) || time.Since(start) >= maxTotal {
break
}
// exponential backoff with jitter, bounded by total budget
backoff := time.Duration(100*(1<<uint(attempt-1))) * time.Millisecond
jitter := time.Duration(rand.Intn(50)) * time.Millisecond
sleep := backoff + jitter
if time.Since(start)+sleep > maxTotal {
break
}
time.Sleep(sleep)
}
return "", lastErr
}
func isTransient(err error) bool {
if err == nil {
return false
}
if strings.Contains(err.Error(), "REAL_SEND_BLOCKED") {
return false
}
if nerr, ok := err.(net.Error); ok && (nerr.Timeout() || nerr.Temporary()) {
return true
}
// best-effort string checks for 5xx/timeout/dns
msg := strings.ToLower(err.Error())
if strings.Contains(msg, "timeout") || strings.Contains(msg, "temporary") || strings.Contains(msg, "dns") {
return true
}
if strings.Contains(msg, " 5") || strings.Contains(msg, "503") || strings.Contains(msg, "502") || strings.Contains(msg, "504") {
return true
}
return false
}
```
**services/messaging-service/internal/handlers/retry_test.go:**
```go
package handlers
import (
"errors"
"net"
"testing"
)
func TestRetrySendNoRetryOnRealSendBlocked(t *testing.T) {
attempts := 0
_, err := retrySend(func() (string, error) {
attempts++
return "", errors.New("REAL_SEND_BLOCKED: set ALLOW_REAL_SEND=true")
})
if err == nil {
t.Fatalf("expected error")
}
if attempts != 1 {
t.Fatalf("expected 1 attempt, got %d", attempts)
}
}
func TestRetrySendRetriesTransientThenSucceeds(t *testing.T) {
attempts := 0
_, err := retrySend(func() (string, error) {
attempts++
if attempts < 3 {
return "", &net.DNSError{IsTemporary: true}
}
return "ok", nil
})
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if attempts != 3 {
t.Fatalf("expected 3 attempts, got %d", attempts)
}
}
```
### 5. Verification Gate
```bash
cd services/messaging-service
go build -o /dev/null cmd/server/main.go && echo "Build PASS"
go test -race ./...
```
**STOP if FAIL. Execute rollback only.**
### 6. Rollback Path
```bash
git checkout HEAD -- services/messaging-service/internal/handlers/sms.go
```
### 7. Artifacts Created/Changed
- `services/messaging-service/internal/handlers/sms.go`
### 8. Git Commit
```bash
git add services/messaging-service/internal/handlers/sms.go services/messaging-service/internal/handlers/retry_test.go
git commit -m "feat: go idempotency + logging (Step 15)"
```
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 15: Implement Go idempotency + logging, build, commit if PASS.
```
---
## Step 16: Implement Go Providers (Like-for-Like)
### 1. Purpose & Pre-requisites
Support all Laravel SMS providers. Use credentials from `sms` or `sms_services`. No hardcoded credentials. Dry-run must not send.
### 2. Files Involved
- `services/messaging-service/internal/providers/router.go`
- `services/messaging-service/internal/providers/*.go`
### 3. Commands to Run
```bash
cd services/messaging-service
go get github.com/twilio/twilio-go@v1.15.0
go get github.com/vonage/vonage-go-sdk@v0.14.1
go get github.com/plivo/plivo-go/v7@v7.29.0
```
### 4. Precise Code Changes
**services/messaging-service/internal/providers/router.go:**
```go
package providers
import (
"errors"
"os"
)
func Send(provider, sid, token, from, baseURL, to, body string) (string, error) {
if os.Getenv("ALLOW_REAL_SEND") != "true" {
return "", errors.New("REAL_SEND_BLOCKED: set ALLOW_REAL_SEND=true")
}
switch provider {
case "twilio":
return SendTwilio(sid, token, from, to, body)
case "nexmo":
return SendNexmo(sid, token, from, to, body)
case "textlocal":
return SendTextLocal(sid, token, from, to, body)
case "plivo":
return SendPlivo(sid, token, from, to, body)
case "signalwire":
return SendSignalwire(sid, token, from, to, body)
case "infobip":
return SendInfobip(sid, baseURL, from, to, body)
case "viber":
return SendViber(token, from, to, body)
case "whatsapp":
return SendWhatsApp(token, from, to, body)
case "telesign":
return SendTelesign(sid, token, from, to, body)
case "sinch":
return SendSinch(sid, token, from, to, body)
case "clickatell":
return SendClickatell(token, from, to, body)
case "mailjet":
return SendMailjet(to, from, body, token)
case "lao":
return SendLao(baseURL, sid, token, from, to, body)
case "aakash":
return SendAakash(sid, from, to, body)
default:
return "", errors.New("unsupported provider: " + provider)
}
}
```
**services/messaging-service/internal/providers/twilio.go:**
```go
package providers
import (
"github.com/twilio/twilio-go"
api "github.com/twilio/twilio-go/rest/api/v2010"
)
func SendTwilio(sid, token, from, to, body string) (string, error) {
client := twilio.NewRestClientWithParams(twilio.ClientParams{
Username: sid,
Password: token,
})
params := &api.CreateMessageParams{}
params.SetTo(to)
params.SetFrom(from)
params.SetBody(body)
resp, err := client.Api.CreateMessage(params)
if err != nil {
return "", err
}
return *resp.Sid, nil
}
```
**services/messaging-service/internal/providers/nexmo.go:**
```go
package providers
import "github.com/vonage/vonage-go-sdk"
func SendNexmo(apiKey, apiSecret, from, to, body string) (string, error) {
auth := vonage.CreateAuthFromKeySecret(apiKey, apiSecret)
client := vonage.NewSMSClient(auth)
resp, _, err := client.Send(from, to, body, vonage.SMSOpts{})
if err != nil {
return "", err
}
if len(resp.Messages) > 0 {
return resp.Messages[0].MessageID, nil
}
return "", nil
}
```
**services/messaging-service/internal/providers/textlocal.go:**
```go
package providers
import (
"net/http"
"net/url"
)
func SendTextLocal(apiKey, _, sender, to, body string) (string, error) {
params := url.Values{
"apikey": {apiKey},
"numbers": {to},
"message": {body},
"sender": {sender},
}
resp, err := httpClient.Get("https://api.textlocal.in/send/?" + params.Encode())
if err != nil {
return "", err
}
defer resp.Body.Close()
return "textlocal-" + to, nil
}
```
**services/messaging-service/internal/providers/plivo.go:**
```go
package providers
import "github.com/plivo/plivo-go/v7"
func SendPlivo(authID, authToken, from, to, body string) (string, error) {
client, err := plivo.NewClient(authID, authToken, &plivo.ClientOptions{})
if err != nil {
return "", err
}
resp, err := client.Messages.Create(plivo.MessageCreateParams{Src: from, Dst: to, Text: body})
if err != nil {
return "", err
}
return resp.MessageUUID, nil
}
```
**services/messaging-service/internal/providers/signalwire.go:**
```go
package providers
import (
"bytes"
"encoding/json"
"fmt"
"net/http"
)
func SendSignalwire(projectID, apiToken, from, to, body string) (string, error) {
url := fmt.Sprintf("https://%s.signalwire.com/api/laml/2010-04-01/Accounts/%s/Messages.json", projectID, projectID)
data := fmt.Sprintf("From=%s&To=%s&Body=%s", from, to, body)
req, _ := http.NewRequest("POST", url, bytes.NewBufferString(data))
req.SetBasicAuth(projectID, apiToken)
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
resp, err := httpClient.Do(req)
if err != nil {
return "", err
}
defer resp.Body.Close()
var result struct{ Sid string `json:"sid"` }
json.NewDecoder(resp.Body).Decode(&result)
return result.Sid, nil
}
```
**services/messaging-service/internal/providers/infobip.go:**
```go
package providers
import (
"bytes"
"encoding/json"
"fmt"
"net/http"
)
func SendInfobip(apiKey, baseURL, from, to, body string) (string, error) {
payload := map[string]interface{}{
"messages": []map[string]interface{}{
{"from": from, "destinations": []map[string]string{{"to": to}}, "text": body},
},
}
jsonData, _ := json.Marshal(payload)
req, _ := http.NewRequest("POST", fmt.Sprintf("https://%s/sms/2/text/advanced", baseURL), bytes.NewBuffer(jsonData))
req.Header.Set("Authorization", "App "+apiKey)
req.Header.Set("Content-Type", "application/json")
resp, err := httpClient.Do(req)
if err != nil {
return "", err
}
defer resp.Body.Close()
var result struct {
Messages []struct {
MessageId string `json:"messageId"`
} `json:"messages"`
}
json.NewDecoder(resp.Body).Decode(&result)
if len(result.Messages) > 0 {
return result.Messages[0].MessageId, nil
}
return "", nil
}
```
**services/messaging-service/internal/providers/viber.go:**
```go
package providers
import (
"bytes"
"encoding/json"
"net/http"
)
func SendViber(token, from, to, body string) (string, error) {
payload := map[string]interface{}{
"receiver": to,
"min_api_version": 1,
"sender": map[string]string{"name": from},
"type": "text",
"text": body,
}
jsonData, _ := json.Marshal(payload)
req, _ := http.NewRequest("POST", "https://chatapi.viber.com/pa/send_message", bytes.NewBuffer(jsonData))
req.Header.Set("X-Viber-Auth-Token", token)
req.Header.Set("Content-Type", "application/json")
resp, err := httpClient.Do(req)
if err != nil {
return "", err
}
defer resp.Body.Close()
return "viber-" + to, nil
}
```
**services/messaging-service/internal/providers/whatsapp.go:**
```go
package providers
import (
"bytes"
"encoding/json"
"net/http"
)
func SendWhatsApp(token, from, to, body string) (string, error) {
payload := map[string]interface{}{
"to": to,
"type": "text",
"text": map[string]string{"body": body},
}
jsonData, _ := json.Marshal(payload)
req, _ := http.NewRequest("POST", "https://graph.facebook.com/v17.0/"+from+"/messages", bytes.NewBuffer(jsonData))
req.Header.Set("Authorization", "Bearer "+token)
req.Header.Set("Content-Type", "application/json")
resp, err := httpClient.Do(req)
if err != nil {
return "", err
}
defer resp.Body.Close()
return "whatsapp-" + to, nil
}
```
**services/messaging-service/internal/providers/telesign.go:**
```go
package providers
import (
"bytes"
"encoding/json"
"fmt"
"net/http"
)
func SendTelesign(customerID, apiKey, from, to, body string) (string, error) {
payload := fmt.Sprintf("phone_number=%s&message=%s&message_type=ARN", to, body)
req, _ := http.NewRequest("POST", "https://rest-api.telesign.com/v1/messaging", bytes.NewBufferString(payload))
req.SetBasicAuth(customerID, apiKey)
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
resp, err := httpClient.Do(req)
if err != nil {
return "", err
}
defer resp.Body.Close()
var result struct{ ReferenceId string `json:"reference_id"` }
json.NewDecoder(resp.Body).Decode(&result)
return result.ReferenceId, nil
}
```
**services/messaging-service/internal/providers/sinch.go:**
```go
package providers
import (
"bytes"
"encoding/json"
"net/http"
)
func SendSinch(apiKey, apiSecret, from, to, body string) (string, error) {
payload := map[string]interface{}{"from": from, "to": []string{to}, "body": body}
jsonData, _ := json.Marshal(payload)
req, _ := http.NewRequest("POST", "https://sms.api.sinch.com/xms/v1/"+apiKey+"/batches", bytes.NewBuffer(jsonData))
req.Header.Set("Authorization", "Bearer "+apiSecret)
req.Header.Set("Content-Type", "application/json")
resp, err := httpClient.Do(req)
if err != nil {
return "", err
}
defer resp.Body.Close()
var result struct{ Id string `json:"id"` }
json.NewDecoder(resp.Body).Decode(&result)
return result.Id, nil
}
```
**services/messaging-service/internal/providers/clickatell.go:**
```go
package providers
import (
"bytes"
"encoding/json"
"net/http"
)
func SendClickatell(apiKey, from, to, body string) (string, error) {
payload := map[string]interface{}{"content": body, "to": []string{to}, "from": from}
jsonData, _ := json.Marshal(payload)
req, _ := http.NewRequest("POST", "https://platform.clickatell.com/messages", bytes.NewBuffer(jsonData))
req.Header.Set("Authorization", apiKey)
req.Header.Set("Content-Type", "application/json")
resp, err := httpClient.Do(req)
if err != nil {
return "", err
}
defer resp.Body.Close()
var result struct {
Messages []struct {
ApiMessageId string `json:"apiMessageId"`
} `json:"messages"`
}
json.NewDecoder(resp.Body).Decode(&result)
if len(result.Messages) > 0 {
return result.Messages[0].ApiMessageId, nil
}
return "", nil
}
```
**services/messaging-service/internal/providers/mailjet.go:**
```go
package providers
import (
"bytes"
"encoding/json"
"net/http"
)
func SendMailjet(to, from, body, token string) (string, error) {
payload := map[string]string{
"Text": to,
"To": body,
"From": from,
}
jsonData, _ := json.Marshal(payload)
req, _ := http.NewRequest("POST", "https://api.mailjet.com/v4/sms-send", bytes.NewBuffer(jsonData))
req.Header.Set("Authorization", "Bearer "+token)
req.Header.Set("Content-Type", "application/json")
resp, err := httpClient.Do(req)
if err != nil {
return "", err
}
defer resp.Body.Close()
return "mailjet-" + to, nil
}
```
**services/messaging-service/internal/providers/lao.go:**
```go
package providers
import (
"net/http"
"net/url"
)
func SendLao(baseURL, apiKey, apiSecret, from, to, body string) (string, error) {
params := url.Values{
"from": {from},
"to": {to},
"message": {body},
"apiKey": {apiKey},
"secret": {apiSecret},
}
resp, err := httpClient.Get(baseURL + "?" + params.Encode())
if err != nil {
return "", err
}
defer resp.Body.Close()
return "lao-" + to, nil
}
```
**services/messaging-service/internal/providers/aakash.go:**
```go
package providers
import (
"net/http"
"net/url"
)
func SendAakash(apiKey, from, to, body string) (string, error) {
params := url.Values{
"auth_token": {apiKey},
"to": {to},
"text": {body},
"from": {from},
}
resp, err := httpClient.Get("https://aakashsms.com/admin/public/sms?" + params.Encode())
if err != nil {
return "", err
}
defer resp.Body.Close()
return "aakash-" + to, nil
}
```
### 5. Verification Gate
```bash
cd services/messaging-service
go build -o /dev/null cmd/server/main.go && echo "Build PASS"
```
**STOP if FAIL. Execute rollback only.**
### 6. Rollback Path
```bash
rm services/messaging-service/internal/providers/*.go
```
### 7. Artifacts Created/Changed
- `services/messaging-service/internal/providers/*`
### 8. Git Commit
```bash
git add services/messaging-service/internal/providers/
git commit -m "feat: add go providers (Step 16)"
```
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 16: Add Go providers + router, build, commit if PASS.
```
---
## Step 17: Add Go Load Test Stub (Fairness Check)
### 1. Purpose & Pre-requisites
Provide a small load test that hits two tenants to verify limiter isolation.
### 2. Files Involved
- `services/messaging-service/cmd/loadtest/main.go`
### 3. Commands to Run
```bash
mkdir -p services/messaging-service/cmd/loadtest
```
### 4. Precise Code Changes
```go
package main
import (
"bytes"
"encoding/json"
"fmt"
"net/http"
"os"
)
type Message struct {
MessageUUID string `json:"message_uuid"`
Recipient string `json:"recipient"`
Body string `json:"body"`
Provider string `json:"provider"`
}
func main() {
url := os.Getenv("URL")
if url == "" {
url = "http://localhost:8080/internal/v1/sms/send-batch"
}
apiKey := os.Getenv("INTERNAL_API_KEY")
if apiKey == "" {
fmt.Println("INTERNAL_API_KEY required")
return
}
makeReq := func(tenantID int, start int) {
msgs := []Message{}
for i := 0; i < 50; i++ {
msgs = append(msgs, Message{
MessageUUID: fmt.Sprintf("t%d-%d", tenantID, start+i),
Recipient: "+1",
Body: "x",
Provider: "twilio",
})
}
payload := map[string]interface{}{
"tenant_id": tenantID,
"messages": msgs,
"dry_run": true,
}
buf, _ := json.Marshal(payload)
req, _ := http.NewRequest("POST", url, bytes.NewBuffer(buf))
req.Header.Set("Content-Type", "application/json")
req.Header.Set("X-Internal-API-Key", apiKey)
resp, err := http.DefaultClient.Do(req)
if err != nil {
fmt.Println("error:", err)
return
}
defer resp.Body.Close()
fmt.Println("tenant", tenantID, "status", resp.StatusCode)
}
makeReq(1, 0)
makeReq(2, 0)
}
```
### 5. Verification Gate
```bash
cd services/messaging-service
INTERNAL_API_KEY=test URL=http://localhost:8080/internal/v1/sms/send-batch go run cmd/loadtest/main.go
```
**STOP if FAIL. Execute rollback only.**
### 6. Rollback Path
```bash
rm services/messaging-service/cmd/loadtest/main.go
```
### 7. Artifacts Created/Changed
- `services/messaging-service/cmd/loadtest/main.go`
### 8. Git Commit
```bash
git add services/messaging-service/cmd/loadtest/main.go
git commit -m "feat: add loadtest stub (Step 17)"
```
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 17: Add Go load test stub, run it against local service, commit if PASS.
```
---
## Step 18: Integrate OFF/SHADOW/ON Modes in SmsController (CANONICAL)
### 0. PRE-INTEGRATION VERIFICATION GATE
**STOP! Do not proceed until this gate passes.**
**Purpose:** Ensure system is clean and Go service is reachable before modifying the controller.
**Commands:**
```bash
# 1. Verify Go Service Health
curl -s -m 2 http://localhost:8080/health || echo "FAIL: Go service down"
# 2. Verify Legacy Send Works (Baseline)
php artisan tinker --execute="
try {
\$log = smsLog(1, '+15550000000', 'Pre-flight check', 'test_gateway');
echo \$log && \$log->id ? 'PASS' : 'FAIL';
} catch (\Exception \$e) {
echo 'FAIL: ' . \$e->getMessage();
}
"
```
**Action:**
- IF **FAIL**: Fix environment or rollback to Step 17. **DO NOT MODIFY SmsController.**
- IF **PASS**: Proceed.
### 1. Purpose & Pre-requisites
Canonical, single-source integration enforcing Invariants 1–4 for OFF/SHADOW/ON without duplicate sends.
### 2. Files Involved
- `app/Http/Controllers/SmsController.php`
### 3. Commands to Run
N/A
### 4. Precise Code Changes
**Add imports at top:**
```php
use App\Models\FeatureFlag;
use App\Services\MessagingServiceClient;
use App\Services\CircuitBreaker;
use Illuminate\Support\Facades\DB;
```
**Insert near start of `campaignSendSms` (before any provider send attempt):**
```php
$tenantId = resolveTenantId();
$mode = FeatureFlag::getMode('messaging_service', $tenantId);
$circuit = new CircuitBreaker('messaging_service');
```
**Generate message_uuid and recipientMap ONCE before any send attempt (Invariant 2):**
```php
// INVARIANT 2: UUID generated ONCE at entry point
$messageUuidMap = [];
$recipientMap = [];
foreach ($campaignSMSs as $sms) {
$uuid = (string) \Str::uuid();
$messageUuidMap[$sms->id] = $uuid;
$recipientMap[$uuid] = '+' . $sms->phones->country_code . $sms->phones->phone;
}
// WARNING: Do NOT generate UUIDs anywhere else in this method
```
**Legacy send isolation (OFF/SHADOW/ON):**
```php
// Wrap the existing provider switch/cases into a single callable to prevent double-send.
$legacySend = function() use ($campaignSMSs, $gateway, $sms_built, $tenantId, $messageUuidMap, $campaign_id) {
// Existing provider switch/cases go here unchanged.
// Ensure each provider path:
// - increments $acceptedCount only on successful send
// - calls smsLog(..., source='laravel')
// - decrements quota ONLY for $acceptedCount > 0
};
// Canonical mode switch (Invariant 1)
switch ($mode) {
case 'OFF':
// Laravel legacy driver sends SMS; Go MUST NOT be called
$legacySend();
return back();
case 'SHADOW':
// Laravel legacy driver sends SMS
$legacySend();
// Go service MUST be called with dry_run=true and MUST NOT affect response
$this->shadowGoSend($tenantId, $messageUuidMap, $recipientMap, $campaign_id, $gateway, $sms_built);
return back();
case 'ON':
// Go service sends SMS (dry_run=false); Laravel MUST NOT send unless Go fails BEFORE acceptance
$goResult = $this->attemptGoSend($tenantId, $messageUuidMap, $recipientMap, $campaign_id, $gateway, $sms_built);
if ($goResult === false) {
// Fallback allowed ONLY on timeout/transport failure/circuit-open
\Log::warning('LEGACY_SEND_FALLBACK_ON_MODE', ['tenant_id' => $tenantId, 'campaign_id' => $campaign_id]);
$legacySend();
}
return back();
}
```
**Helper Methods (add to `SmsController`):**
```php
private function shadowGoSend(int $tenantId, array $messageUuidMap, array $recipientMap, int $campaignId, string $gateway, $smsBuilt): void
{
$circuit = new CircuitBreaker('messaging_service');
if ($circuit->allowRequest()) {
try {
$client = new MessagingServiceClient();
$messages = [];
foreach ($messageUuidMap as $smsId => $uuid) {
$messages[] = [
'message_uuid' => $uuid,
'recipient' => $recipientMap[$uuid],
'body' => strip_tags($smsBuilt->body),
'provider' => $gateway
];
}
$client->sendBatch($tenantId, $messages, $campaignId, true);
$circuit->recordSuccess();
} catch (\Exception $e) {
$circuit->recordFailure();
\Log::warning('Messaging service (SHADOW) failed', ['error' => $e->getMessage()]);
}
}
}
private function attemptGoSend(int $tenantId, array $messageUuidMap, array $recipientMap, int $campaignId, string $gateway, $smsBuilt): bool
{
$circuit = new CircuitBreaker('messaging_service');
if (!$circuit->allowRequest()) {
return false;
}
try {
$client = new MessagingServiceClient();
$messages = [];
foreach ($messageUuidMap as $smsId => $uuid) {
$messages[] = [
'message_uuid' => $uuid,
'recipient' => $recipientMap[$uuid],
'body' => strip_tags($smsBuilt->body),
'provider' => $gateway
];
}
$result = $client->sendBatch($tenantId, $messages, $campaignId, false);
$circuit->recordSuccess();
// INVARIANT 4: Quota decrement only for accepted OR duplicate status in ON mode
$accepted = array_filter($result['results'] ?? [], function ($r) {
return in_array($r['status'] ?? '', ['accepted', 'duplicate']);
});
DB::transaction(function () use ($accepted, $recipientMap, $campaignId, $smsBuilt, $gateway, $tenantId) {
foreach ($accepted as $r) {
$recipient = $recipientMap[$r['message_uuid']] ?? '';
if ($recipient === '') {
\Log::error('ON mode: missing recipient for uuid ' . $r['message_uuid']);
continue;
}
smsLog(
$campaignId,
$recipient,
strip_tags($smsBuilt->body),
$gateway,
$r['message_uuid'],
'go',
$tenantId,
$gateway,
false,
$r['status'] ?? 'accepted',
$r['provider_message_id'] ?? null,
$r['error_code'] ?? null
);
}
$acceptedCount = count($accepted);
if ($acceptedCount > 0) {
EmailSMSLimitRate::where('owner_id', $tenantId)->decrement('sms', $acceptedCount);
}
});
\Log::info('LEGACY_SEND_SKIPPED_ON_MODE', ['tenant_id' => $tenantId, 'campaign_id' => $campaignId]);
return true;
} catch (\Exception $e) {
$circuit->recordFailure();
\Log::warning('Messaging service (ON) failed', ['error' => $e->getMessage()]);
return false;
}
}
```
**Legacy send (OFF or fallback):**
- Initialize `$acceptedCount = 0;` per provider case.
- Increment only after successful send.
- Log with `smsLog(...)` using `$messageUuidMap` and `source='laravel'`.
**Example replacement inside each provider loop:**
```php
$acceptedCount = 0;
foreach ($campaignSMSs as $campaignSMS) {
// existing provider send call
$acceptedCount++;
smsLog(
$campaignSMS->id,
$campaignSMS->phones->phone,
strip_tags($sms_built->body),
$gateway,
$messageUuidMap[$campaignSMS->id],
'laravel',
$tenantId,
$gateway,
false,
'accepted',
null,
null
);
}
if ($acceptedCount > 0) {
EmailSMSLimitRate::where('owner_id', $tenantId)->decrement('sms', $acceptedCount);
}
```
**Mandatory Rules:**
- OFF: Laravel local driver only.
- SHADOW: Laravel local send + Go dry_run (no provider calls in Go). Go errors do not affect user response.
- ON: Go send; fallback to legacy when Go fails, 5xx, network error, or >500ms timeout.
- Quota decrement only for accepted/duplicate sends in ON mode.
- Recipient for ON logging MUST come from request payload, not response.
### 5. Verification Gate
```bash
# Simulate Go down and verify legacy send still works (fallback path)
php artisan sms:verify-fallback
# Verify 500ms timeout triggers fallback (use slow endpoint)
php artisan tinker --execute="
config(['services.messaging.url' => 'http://httpbin.org/delay/1']);
\$client = new App\Services\MessagingServiceClient();
try {
\$client->sendBatch(1, [['message_uuid'=>'test-'.time(),'recipient'=>'+1','body'=>'x','provider'=>'twilio']], null, true);
echo 'FAIL: should have timed out';
} catch (\Illuminate\Http\Client\ConnectionException \$e) {
echo 'PASS: timeout triggered';
}
"
# Force Go down by pointing to invalid URL, then trigger ON-mode send and verify fallback marker
# (Set in .env for this step only, then revert)
# MESSAGING_SERVICE_URL=http://127.0.0.1:59999
# Expect log marker: LEGACY_SEND_FALLBACK_ON_MODE
grep -n "LEGACY_SEND_FALLBACK_ON_MODE" storage/logs/laravel.log
# Simulate Go healthy and verify legacy is NOT called in ON mode
# Expect log marker: LEGACY_SEND_SKIPPED_ON_MODE and NO LEGACY_SEND_FALLBACK_ON_MODE
grep -n "LEGACY_SEND_SKIPPED_ON_MODE" storage/logs/laravel.log
! grep -n "LEGACY_SEND_FALLBACK_ON_MODE" storage/logs/laravel.log
# Quota invariant checks (SQL)
# SELECT sms FROM email_s_m_s_limit_rates WHERE owner_id = <tenant_id>;
# 1) SHADOW mode must not decrement (run a SHADOW send between before/after)
# 2) ON mode must decrement only accepted count (run an ON send between before/after)
php artisan tinker --execute="
\$tenantId = 1;
\$before = DB::table('email_s_m_s_limit_rates')->where('owner_id', \$tenantId)->value('sms');
echo 'before='.\$before.PHP_EOL;
"
# Trigger send via existing campaign route (see Step 20 for route/ids)
php artisan tinker --execute="
\$tenantId = 1;
\$after = DB::table('email_s_m_s_limit_rates')->where('owner_id', \$tenantId)->value('sms');
echo 'after='.\$after.PHP_EOL;
"
# Simulate DB failure during transaction and verify no partial source='go' rows + no quota decrement
# (Example: temporarily throw inside transaction block and confirm rollback)
# SELECT COUNT(*) FROM sms_logs WHERE source='go' AND created_at >= NOW() - INTERVAL 5 MINUTE;
```
**STOP if FAIL. Execute rollback only.**
### 6. Rollback Path
```bash
git checkout HEAD -- app/Http/Controllers/SmsController.php
```
### 7. Artifacts Created/Changed
- `app/Http/Controllers/SmsController.php`
### 8. Git Commit
```bash
git add app/Http/Controllers/SmsController.php
git commit -m "feat: integrate shadow/on modes + uuid logging (Step 18)"
```
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 18: Update SmsController for OFF/SHADOW/ON, verify gate, commit if PASS.
```
---
## Step 19: UUID-based Reconciliation Gate
### 1. Purpose & Pre-requisites
Run a deterministic UUID-based reconciliation between `sms_logs` and `go_sms_logs` after dual-run (Invariant 3).
### 2. Files Involved
Database state only.
### 3. Commands to Run
**Join Key:** `message_uuid` is the ONLY join key for reconciliation.
**Mandatory Verification Queries:**
```sql
-- Laravel side
SELECT message_uuid FROM sms_logs WHERE source='go';
-- Go side
SELECT message_uuid FROM go_sms_logs;
```
**Mismatch Detection Rule:**
A mismatch exists when:
1. `message_uuid` exists in `sms_logs` (source='go') but NOT in `go_sms_logs` → `missing_in_go`
2. `message_uuid` exists in `go_sms_logs` but NOT in `sms_logs` (source='go') → `missing_in_laravel`
3. `status` values differ between matched rows → `status_mismatch`
4. `provider_message_id` values differ between matched rows → `provider_message_id_mismatch`
```bash
php artisan tinker --execute="
\$rows = DB::select(\"
SELECT l.message_uuid,
l.status AS laravel_status,
g.status AS go_status,
l.provider_message_id AS laravel_provider_id,
g.provider_message_id AS go_provider_id,
CASE
WHEN g.message_uuid IS NULL THEN 'missing_in_go'
WHEN l.status <> g.status THEN 'status_mismatch'
WHEN l.provider_message_id <> g.provider_message_id THEN 'provider_message_id_mismatch'
ELSE 'match'
END AS notes
FROM sms_logs l
LEFT JOIN go_sms_logs g ON l.message_uuid = g.message_uuid
WHERE l.source = 'go'
AND (g.message_uuid IS NULL
OR l.status <> g.status
OR l.provider_message_id <> g.provider_message_id)
ORDER BY l.created_at DESC
LIMIT 100;
\");
print_r(\$rows);
"
```
### 4. Precise Code Changes
None (query-only gate).
### 5. Verification Gate
- PASS if query returns 0 rows.
- FAIL if any row is returned. **STOP and execute rollback only.**
**Report Template (Markdown):**
```
| message_uuid | laravel_status | go_status | laravel_provider_id | go_provider_id | notes |
| --- | --- | --- | --- | --- | --- |
| <uuid> | <status> | <status> | <id> | <id> | <reason> |
```
### 6. Rollback Path
```bash
php artisan tinker --execute="
App\Models\FeatureFlag::where('tenant_id',1)->update(['mode'=>'OFF']);
"
# Re-run reconciliation to confirm no new mismatches accumulate
php artisan tinker --execute="
\$rows = DB::select(\"
SELECT l.message_uuid
FROM sms_logs l
LEFT JOIN go_sms_logs g ON l.message_uuid = g.message_uuid
WHERE l.source = 'go'
AND (g.message_uuid IS NULL
OR l.status <> g.status
OR l.provider_message_id <> g.provider_message_id)
LIMIT 1;
\");
echo count(\$rows) === 0 ? 'PASS' : 'FAIL';
"
```
### 7. Artifacts Created/Changed
None (DB-only verification).
### 8. Git Commit
N/A
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 19: Run UUID-based reconciliation SQL gate; STOP if mismatches; rollback by disabling tenant.
```
---
## Step 20: End-to-End SHADOW Mode Dual-Run Verification
### 1. Purpose & Pre-requisites
Prove SHADOW mode performs legacy send plus Go dry_run, with matching message_uuid in both logs.
### 2. Files Involved
Database state only.
### 3. Commands to Run
```bash
# Enable SHADOW for test tenant
php artisan tinker --execute="
App\Models\FeatureFlag::updateOrCreate(
['tenant_id' => 1, 'feature_name' => 'messaging_service'],
['mode' => 'SHADOW']
);
Cache::forget('feature_flag:messaging_service:tenant:1');
"
# Find a campaign with sms_template_id and its gateway (uses existing data)
php artisan tinker --execute="
\$c = App\Models\Campaign::whereNotNull('sms_template_id')->first();
if (!\$c) { echo 'NO_CAMPAIGN_WITH_SMS_TEMPLATE'; exit(1); }
\$gateway = optional(\$c->relationWithSMSServer)->sms_name;
if (!\$gateway) { echo 'NO_GATEWAY_FOR_CAMPAIGN'; exit(1); }
echo \$c->id.' '.\$c->sms_template_id.' '.\$gateway;
"
# Send via existing route (from routes/sms.php):
# GET /campaign/send-sms/campaign-{campaign_id}/{sms_template_id}/{gateway}
curl -s \"http://localhost/campaign/send-sms/campaign-<campaign_id>/<sms_template_id>/<gateway>\"
```
### 4. Precise Code Changes
None (verification only).
### 5. Verification Gate
```sql
SELECT message_uuid FROM sms_logs ORDER BY id DESC LIMIT 1;
SELECT message_uuid FROM go_sms_logs ORDER BY id DESC LIMIT 1;
SELECT dry_run FROM go_sms_logs ORDER BY id DESC LIMIT 1;
```
**PASS if:**
- `sms_logs.message_uuid` is not null
- `go_sms_logs.message_uuid` matches `sms_logs.message_uuid`
- `go_sms_logs.dry_run = 1`
**STOP if FAIL. Execute rollback only.**
### 6. Rollback Path
```bash
php artisan tinker --execute="App\Models\FeatureFlag::where('tenant_id',1)->update(['mode'=>'OFF']);"
```
### 7. Artifacts Created/Changed
None (DB-only verification).
### 8. Git Commit
N/A
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 20: Enable SHADOW, send via campaign route, verify matching UUIDs + dry_run=1, rollback if FAIL.
```
---
## Step 21: Implement SMS Reconciliation Command
### 1. Purpose & Pre-requisites
Compare Laravel `sms_logs` with Go `go_sms_logs` using `message_uuid` as the ONLY join key (Invariant 3), and detect missing rows and mismatches.
### 2. Files Involved
- `app/Console/Commands/ReconcileSmsLogs.php`
### 3. Commands to Run
```bash
php artisan make:command ReconcileSmsLogs
```
### 4. Precise Code Changes
```php
<?php
namespace App\Console\Commands;
use Illuminate\Console\Command;
use Illuminate\Support\Facades\DB;
class ReconcileSmsLogs extends Command
{
protected $signature = 'sms:reconcile {--date=today}';
protected $description = 'Reconcile sms_logs vs go_sms_logs by message_uuid';
public function handle()
{
$date = $this->option('date') === 'today' ? now()->toDateString() : $this->option('date');
$laravel = DB::table('sms_logs')
->whereDate('created_at', $date)
->select('message_uuid','status','provider_message_id')
->get()
->keyBy('message_uuid');
$go = DB::table('go_sms_logs')
->whereDate('created_at', $date)
->select('message_uuid','status','provider_message_id')
->get()
->keyBy('message_uuid');
$mismatches = [];
foreach ($laravel as $uuid => $row) {
if (!$go->has($uuid)) {
$mismatches[] = ['message_uuid' => $uuid, 'reason' => 'missing_in_go'];
continue;
}
$g = $go[$uuid];
if ($row->status !== $g->status) {
$mismatches[] = ['message_uuid' => $uuid, 'reason' => 'status_mismatch'];
}
if ($row->provider_message_id !== $g->provider_message_id) {
$mismatches[] = ['message_uuid' => $uuid, 'reason' => 'provider_message_id_mismatch'];
}
}
foreach ($go as $uuid => $row) {
if (!$laravel->has($uuid)) {
$mismatches[] = ['message_uuid' => $uuid, 'reason' => 'missing_in_laravel'];
}
}
$this->info("Compared: " . max($laravel->count(), $go->count()));
$this->info("Mismatches: " . count($mismatches));
foreach ($mismatches as $m) {
$this->line($m['message_uuid'] . " " . $m['reason']);
}
return count($mismatches) === 0 ? 0 : 1;
}
}
```
### 5. Verification Gate
```bash
php artisan sms:reconcile
```
**Expected output example:**
```
Compared: 1000
Mismatches: 0
Exit Code: 0
```
**STOP if FAIL. Execute rollback only.**
### 6. Rollback Path
```bash
rm app/Console/Commands/ReconcileSmsLogs.php
```
### 7. Artifacts Created/Changed
- `app/Console/Commands/ReconcileSmsLogs.php`
### 8. Git Commit
```bash
git add app/Console/Commands/ReconcileSmsLogs.php
git commit -m "feat: add sms reconciliation command (Step 21)"
```
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 21: Add ReconcileSmsLogs command, run php artisan sms:reconcile, commit if PASS.
```
---
## Step 22: Redis Session Store + Internal Session Introspection
### 1. Purpose & Pre-requisites
Enable shared sessions via Redis and provide `/internal/session/introspect` for Spring Boot to query authenticated session data. Avoid PHP<->Java deserialization.
### 2. Files Involved
- `config/session.php`
- `.env`, `.env.example`
- `routes/web.php`
- `app/Http/Controllers/InternalSessionController.php`
### 3. Commands to Run
```bash
php artisan make:controller InternalSessionController
```
### 4. Precise Code Changes
**.env:**
```
SESSION_DRIVER=redis
SESSION_CONNECTION=default
SESSION_STORE=default
```
**.env.example:**
```
SESSION_DRIVER=redis
SESSION_CONNECTION=default
SESSION_STORE=default
```
**routes/web.php:**
```php
Route::post('/internal/session/introspect', [App\Http\Controllers\InternalSessionController::class, 'introspect']);
```
**app/Http/Controllers/InternalSessionController.php:**
```php
<?php
namespace App\Http\Controllers;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\Auth;
class InternalSessionController extends Controller
{
public function introspect(Request $request)
{
if (!Auth::check()) {
return response()->json(['error' => 'unauthorized'], 401);
}
$user = Auth::user();
return response()->json([
'user_id' => $user->id,
'tenant_id' => resolveTenantId(),
'user_type' => $user->user_type,
'permissions' => $user->permissions ?? [],
]);
}
}
```
### 5. Verification Gate
```bash
php artisan tinker --execute="echo config('session.driver') === 'redis' ? 'PASS' : 'FAIL';"
```
**STOP if FAIL. Execute rollback only.**
### 6. Rollback Path
```bash
git checkout HEAD -- config/session.php routes/web.php app/Http/Controllers/InternalSessionController.php
```
### 7. Artifacts Created/Changed
- `config/session.php`
- `.env`, `.env.example`
- `routes/web.php`
- `app/Http/Controllers/InternalSessionController.php`
### 8. Git Commit
```bash
git add config/session.php routes/web.php app/Http/Controllers/InternalSessionController.php .env.example
git commit -m "feat: redis session + introspect endpoint (Step 22)"
```
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 22: Add Redis session config + internal introspect endpoint, verify PASS, commit if PASS.
```
---
## Step 23: Docker Deployment for Go Service
### 1. Purpose & Pre-requisites
Production deployment packaging.
### 2. Files Involved
- `services/messaging-service/Dockerfile`
- `services/messaging-service/docker-compose.yml`
### 3. Commands to Run
N/A
### 4. Precise Code Changes
**services/messaging-service/Dockerfile:**
```dockerfile
FROM golang:1.21-alpine AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o server cmd/server/main.go
FROM alpine:3.18
RUN apk --no-cache add ca-certificates
WORKDIR /app
COPY --from=builder /app/server .
EXPOSE 8080
CMD ["./server"]
```
**services/messaging-service/docker-compose.yml:**
```yaml
version: '3.8'
services:
messaging-service:
build: .
ports:
- "8080:8080"
environment:
- PORT=8080
- INTERNAL_API_KEY=${INTERNAL_API_KEY}
- DB_DSN=${DB_DSN}
- ALLOW_REAL_SEND=${ALLOW_REAL_SEND:-false}
- WORKERS=${WORKERS:-20}
- TENANT_RPS=${TENANT_RPS:-10}
- TENANT_BURST=${TENANT_BURST:-20}
```
### 5. Verification Gate
```bash
cd services/messaging-service
docker build -t messaging-service:test . && echo "Docker Build PASS"
```
**STOP if FAIL. Execute rollback only.**
### 6. Rollback Path
```bash
rm services/messaging-service/Dockerfile services/messaging-service/docker-compose.yml
```
### 7. Artifacts Created/Changed
- `services/messaging-service/Dockerfile`
- `services/messaging-service/docker-compose.yml`
### 8. Git Commit
```bash
git add services/messaging-service/Dockerfile services/messaging-service/docker-compose.yml
git commit -m "feat: add docker deployment (Step 23)"
```
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 23: Add Dockerfile + compose, build, commit if PASS.
```
---
## Step 24: Tenant Rollout (SHADOW)
### 1. Purpose & Pre-requisites
Enable SHADOW mode for a single test tenant.
### 2. Files Involved
Database state only.
### 3. Commands to Run
```bash
php artisan tinker --execute="
App\Models\FeatureFlag::updateOrCreate(
['tenant_id' => 1, 'feature_name' => 'messaging_service'],
['mode' => 'SHADOW']
);
Cache::forget('feature_flag:messaging_service:tenant:1');
echo 'Tenant 1 set to SHADOW';
"
```
### 4. Precise Code Changes
None (DB only).
### 5. Verification Gate
```bash
php artisan tinker --execute="
\$mode = App\Models\FeatureFlag::getMode('messaging_service', 1);
echo \$mode === 'SHADOW' ? 'PASS' : 'FAIL';
"
```
**STOP if FAIL. Execute rollback only.**
### 6. Rollback Path
```bash
php artisan tinker --execute="App\Models\FeatureFlag::where('tenant_id',1)->update(['mode'=>'OFF']); Cache::forget('feature_flag:messaging_service:tenant:1');"
```
### 7. Artifacts Created/Changed
- Database: `feature_flags` record
### 8. Git Commit
N/A
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 24: Enable SHADOW for tenant 1, verify mode, rollback if FAIL.
```
---
## Step 25: Tenant Rollout (ON)
### 1. Purpose & Pre-requisites
Enable ON mode for test tenant.
### 2. Files Involved
Database state only.
### 3. Commands to Run
```bash
php artisan tinker --execute="
App\Models\FeatureFlag::where('tenant_id', 1)->update(['mode' => 'ON']);
Cache::forget('feature_flag:messaging_service:tenant:1');
echo 'Tenant 1 set to ON';
"
```
### 4. Precise Code Changes
None.
### 5. Verification Gate
```bash
php artisan tinker --execute="echo App\Models\FeatureFlag::getMode('messaging_service', 1);"
php artisan sms:reconcile
```
**STOP if FAIL. Execute rollback only.**
### 6. Rollback Path
```bash
php artisan tinker --execute="App\Models\FeatureFlag::where('tenant_id',1)->update(['mode'=>'SHADOW']); Cache::forget('feature_flag:messaging_service:tenant:1');"
```
### 7. Artifacts Created/Changed
- Database: `feature_flags` record
### 8. Git Commit
N/A
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 25: Enable ON for tenant 1, run reconcile, rollback if FAIL.
```
---
## Step 26: Global Rollout
### 1. Purpose & Pre-requisites
Enable ON mode globally after successful testing.
### 2. Files Involved
Database state only.
### 3. Commands to Run
```bash
php artisan tinker --execute="
App\Models\FeatureFlag::whereNull('tenant_id')
->where('feature_name', 'messaging_service')
->update(['mode' => 'ON']);
Cache::forget('feature_flag:messaging_service:tenant:1');
echo 'Global default set to ON';
"
```
### 4. Precise Code Changes
None.
### 5. Verification Gate
```bash
php artisan sms:reconcile
```
**STOP if FAIL. Execute rollback only.**
### 6. Rollback Path
```bash
php artisan tinker --execute="
App\Models\FeatureFlag::where('feature_name', 'messaging_service')->update(['mode'=>'OFF']);
Cache::forget('feature_flag:messaging_service:tenant:1');
"
```
### 7. Artifacts Created/Changed
- Database: `feature_flags` global record
### 8. Git Commit
N/A
### 9. Agent Prompt (Copy/Paste)
```
Execute Step 26: Enable ON globally, run reconcile, rollback if FAIL.
```
---
# APPENDIX: QUICK ROLLBACK
**Single tenant:**
```sql
UPDATE feature_flags SET mode='OFF' WHERE tenant_id = <id>;
```
**Global:**
```sql
UPDATE feature_flags SET mode='OFF' WHERE feature_name='messaging_service';
```
**END OF RUNBOOK**
NovaShell
🛸 NovaShell — Cyber Yellow Mode