> ## Documentation Index
> Fetch the complete documentation index at: https://ihub.learnille.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Learnille Impression Tracking System — PRD

> Migrated from the previous Learnille internal documentation hub

## 1. Overview

This document defines the Product Requirements for a secure, tamper-resistant impression tracking system for Learnille's marketplace. The system tracks course and consultation visibility, enables instructor/consultant analytics, and supports relevance ranking.

***

## 2. Goals

### Primary Goals

* Track **true impressions**: product actually appears in user viewport
* Maintain **tamper-resistant** impression signals via HMAC signing
* Provide accurate **session-level deduping** and **per-product metrics**
* Support **obfuscated endpoints** to prevent competitor detection

### Secondary Goals

* Enable instructor/consultant dashboards (impressions, CTR, referrers)
* Support ML/relevance ranking via impression data
* Allow anti-fraud heuristics (velocity, bot detection)

***

## 3. Non-Goals

* No external analytics vendor integration
* No third-party fingerprinting SDK
* No cookie consent requirements (privacy-first approach)
* No real-time streaming infrastructure at initial stage

***

## 4. Architecture Overview

### Initial Phase

```
Client (Public Web)
   ↓
Beacon Endpoint (NestJS API)
   ↓
BeaconGuard (HMAC + Timestamp + Nonce)
   ↓
ImpressionService
   ↓
PostgreSQL (impressions)
   ↓
Redis (nonce tracking, rate limiting)
```

### Future Upgrade Path

```
Client → API → Redis Streams → Worker → PostgreSQL (TimescaleDB) → Aggregations
```

* **TimescaleDB**: Convert `impressions` table to a hypertable for efficient time-series partitioning and faster aggregation queries (e.g., `time_bucket`).

***

## 5. Client-Side Requirements

### 5.1 Impression Trigger

Use browser `IntersectionObserver` to record an impression only when:

1. Product card enters viewport
2. Remains visible ≥ 150ms
3. Impression for (entity\_id + session\_id) has not been fired this session

### 5.2 Required Client Fields

| Field | Type | Description |
| - | - | - |
| `entity_id` | UUID | Course or consultation ID |
| `entity_type` | string | "course", "consultation", "instructor", "consultant" |
| `event_type` | string | "impression", "click", "bounce" |
| `session_id` | UUID | Browser session identifier |
| `referrer` | string | Document referrer URL |
| `device_fingerprint` | string | Browser fingerprint hash |
| `timestamp` | number | Unix timestamp (ms) |
| `nonce` | string | 12-char random string |
| `signature` | string | HMAC-SHA256 signature |

### 5.3 Legitimacy Verification (Handshake)

To ensure requests are absolutely legitimate and "authorized" by the server without user login:

**1. Handshake Phase**

* Client calls `GET /api/v1/beacon/h`.
* Server verifies client (IP reputation, standard headers).
* Server returns a **Signed Session Token** (JWT-like) containing:
  * `session_id`
  * `issued_at`
  * `expiry` (e.g., 2 hours)
* Token is signed with `BEACON_PRIVATE_KEY`.

**2. Signing Phase**
Client signs each impression batch using:

```
HMAC_SHA256(payload + timestamp + nonce + session_token, BEACON_SESSION_SECRET)
```

* `BEACON_SESSION_SECRET` is derived from the session token or a shared secret.
* Server verifies both the **Session Token signature** and the **Impression HMAC**.

This "double-seal" ensures:

* The client was initialized by our server.
* The payload hasn't been tampered with.
* Replay attacks are impossible (even if signature is stolen, it's tied to an expired token).

***

### 5.4 Payload Obfuscation

All fields use short codes to obscure intent:

| Short | Full |
| - | - |
| `t` | entity\_type |
| `e` | entity\_id |
| `v` | event\_type |
| `r` | referrer |
| `f` | device\_fingerprint |
| `s` | session\_id |

Entity types use codes: `c`=course, `n`=consultation, `i`=instructor, `o`=consultant

***

## 6. Session Requirements

### 6.1 Browser Session ID

* Stored in `sessionStorage` (not cookies)
* Value: UUIDv4
* Expires when tab closes
* Key: `bkn_sid`

### 6.2 Deduplication

* Track fired impressions in `sessionStorage`
* Key format: `bkn:{entity_type}:{entity_id}`
* Prevents duplicate tracking per session

### 6.3 Device Fingerprint

Minimal fingerprint using:

* User agent
* Screen resolution
* Timezone offset
* Language

Hash via: `SHA256(ua + screen + tz + lang).substring(0, 16)`

***

### 6.4 Client-Side Batching

To improve performance and reduce server load:

* **Queue**: Store impressions in a memory queue.
* **Flush Triggers**:
  * Time: Every 2 seconds.
  * Size: Every 10 queued items.
  * Event: `pagehide` / `beforeunload` (using `navigator.sendBeacon`).
* **Format**: Batch array sent to single endpoint.

***

## 7. Server Endpoint Specification

### 7.1 HTTP (Handshake)

```
GET /api/v1/beacon/h
```

Returns: `{ "token": "signed.session.token", "key": "temporary-signing-key" }`

### 7.2 HTTP (Batch)

```
POST /api/v1/beacon/batch
Content-Type: application/json
```

**Request Body**:

```json theme={null}
{
  "token": "signed.session.token",
  "batch": [
    { "d": "...", "ts": 1702..., "n": "...", "sig": "..." }
  ]
}
```

### 7.3 HTTP (Pixel Fallback)

(Unchanged: Single request via `px.gif`)

***

## 8. Attribution ("The Golden Thread")

To track conversion value back to impressions:

### 8.1 Search Session Correlation

* Generate `search_session_id` on search execution.
* Store `search_session_id` in `sessionStorage` for persistence across the search results page.
* Pass `search_session_id` to:
  1. Impression payload (`ss` field).
  2. Product URL (query param `?sid=...`).

### 8.2 Checkout Association

* The `useBeacon` hook captures `sid` from the URL on product detail pages and saves it to `sessionStorage` (key: `search_session_id`).
* On `Order` creation, the frontend sends this `search_session_id` to the backend.
* The `Order` entity persists `searchSessionId`.
* **Analytics**: Join `orders` table with `impressions` via `search_session_id` to calculate "Revenue per Search" and "Conversion Rate by Rank".

***

## 9. Dedupe Logic

### 8.1 Client-Side Dedupe

Per session, per entity:

```
sessionStorage.getItem(`bkn:${type}:${id}`)
```

### 8.2 Server-Side Dedupe

Redis-based with 1-hour TTL:

```
impression:{user_or_ip}:{entity_type}:{entity_id}
```

Uses `SET NX` (set-if-not-exists) for atomic deduplication.

***

## 9. Database Schema

### impressions

```sql theme={null}
CREATE TABLE impressions (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  entity_type VARCHAR(50) NOT NULL,
  entity_id UUID NOT NULL,
  event_type VARCHAR(50) NOT NULL,
  user_id UUID,
  session_id VARCHAR(255),
  ip_address INET,
  device_type VARCHAR(100),
  device_fingerprint VARCHAR(255),
  referrer TEXT,
  search_query TEXT,
  value DECIMAL(10,2),
  metadata JSONB,
  created_at TIMESTAMPTZ DEFAULT NOW()
);

CREATE INDEX idx_impressions_entity ON impressions(entity_type, entity_id, created_at);
CREATE INDEX idx_impressions_user ON impressions(user_id, created_at);
CREATE INDEX idx_impressions_session ON impressions(session_id);
```

### Metadata JSONB

Stores owner IDs for filtering:

```json theme={null}
{
  "instructor_id": "uuid",
  "consultant_id": "uuid"
}
```

***

## 10. Analytics Queries

### Impressions by Period

```sql theme={null}
SELECT 
  DATE_TRUNC('day', created_at) AS period,
  COUNT(*) AS impressions
FROM impressions
WHERE entity_type = 'course' 
  AND entity_id = :entityId
  AND created_at >= NOW() - INTERVAL '7 days'
GROUP BY period
ORDER BY period;
```

### CTR Calculation

```sql theme={null}
SELECT 
  SUM(CASE WHEN event_type = 'click' THEN 1 ELSE 0 END)::float /
  NULLIF(SUM(CASE WHEN event_type = 'impression' THEN 1 ELSE 0 END), 0) AS ctr
FROM impressions
WHERE entity_type = 'course' AND entity_id = :entityId;
```

### Instructor Dashboard

```sql theme={null}
SELECT 
  entity_id,
  COUNT(*) FILTER (WHERE event_type = 'impression') AS impressions,
  COUNT(*) FILTER (WHERE event_type = 'click') AS clicks
FROM impressions
WHERE entity_type = 'course'
  AND metadata->>'instructor_id' = :instructorId
  AND created_at >= NOW() - INTERVAL '30 days'
GROUP BY entity_id;
```

***

## 11. Fraud & Abuse Detection

### Early Indicators

| Signal | Threshold | Action |
| - | - | - |
| Excessive impressions/min | >100 per session | Reject (HTTP 403 / Forbidden) |
| Invalid signature | Any | Reject |
| Expired timestamp | >30s | Reject |
| Duplicate nonce | Any | Reject |
| Same IP, many entities | >50 in 1 min | Flag |

### Future Upgrades

* Browser behavior scoring
* ML-based anomaly detection
* Scroll velocity analysis

***

## 12. Operational Requirements

### 12.1 Performance

* API handles ≥500 req/sec
* ≤15ms average response time
* Redis for hot-path operations

### 12.2 Security

* Rotate BEACON\_SECRET every 90 days
* Enforce TLS
* Limit body size to 4KB
* Rate-limit by IP/session

### 12.3 Monitoring

* Track rejection rate by reason
* Alert on >5% signature failures
* Log suspicious patterns

***

## 13. Client SDK Requirements

### useBeacon Hook

```typescript theme={null}
function useBeacon(entityType: string, entityId: string) {
  useEffect(() => {
    if (!entityId) return;
    const key = `bkn:${entityType}:${entityId}`;
    if (sessionStorage.getItem(key)) return;
    sessionStorage.setItem(key, '1');
    beacon(entityType, entityId, 'impression');
  }, [entityType, entityId]);
}
```

### beacon Function

* Generate/maintain session ID
* Sign events with HMAC
* Fallback to pixel if fetch blocked
* Batch send (via BeaconQueue)

***

## 14. Success Metrics

| Metric | Target |
| - | - |
| Impression delivery rate | ≥98% |
| Server processing time | ≤15ms |
| Signature rejection rate | \<1% |
| Duplicate detection rate | >95% |
| CTR accuracy | ±2% |

***

## 15. Environment Variables

| Variable | Location | Description |
| - | - | - |
| `BEACON_SECRET` | Server | HMAC signing key (32+ chars) |
| `NEXT_PUBLIC_BEACON_KEY` | Frontend | Public signing key |
| `BEACON_RATE_LIMIT` | Server | Requests per minute (default: 30) |
| `BEACON_NONCE_TTL` | Server | Nonce expiry seconds (default: 60) |

***

## 16. File Structure

```
server/src/impression/
├── impression.module.ts
├── impression.service.ts
├── impression.controller.ts
├── beacon.controller.ts     # Public obfuscated endpoints
├── beacon.service.ts        # Decoding & batching logic
├── guards/
│   └── beacon.guard.ts      # HMAC/Timestamp validation
└── dto/
    └── beacon.dto.ts        # Signed payload definitions

public/src/
├── services/beacon.ts
└── hooks/useBeacon.ts
```

***

## 17. Implementation Phases

### Phase 1: Core (Week 1)

* [ ] BeaconController with endpoints
* [ ] BeaconGuard with HMAC validation
* [ ] Frontend beacon.ts service
* [ ] useBeacon hook
* [ ] Integration on course/consultation pages

### Phase 2: Security (Week 2)

* [ ] Timestamp validation
* [ ] Nonce tracking in Redis
* [ ] Rate limiting
* [ ] Behavioral detection

### Phase 3: Analytics (Week 3)

* [ ] Dashboard API endpoints
* [ ] Aggregation queries
* [ ] Instructor/consultant views

***

## 18. Open Questions

1. Should we track scroll depth for engagement scoring?
2. Should impressions be versioned for ranking model training?
3. Should we implement A/B testing for different CTR thresholds?

***

## 19. Next Steps

1. ✅ Approve PRD
2. Generate API contract
3. Generate NestJS implementation
4. Generate JS client SDK
5. Deploy MVP
6. Add monitoring & alerts


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.