feat: добавить базовые сервисы image-platform
- добавлены backend, admin, gateway и worker skeleton - добавлены Drizzle schema, database package и initial migration - добавлены shared packages для RabbitMQ topology и S3 helpers - обновлены dev-инфраструктура, env example, scripts и dependencies - обновлена документация под versioned image URLs и read-through flow
This commit is contained in:
33
.env.example
33
.env.example
@@ -19,12 +19,39 @@ S3_SECRET_ACCESS_KEY=image-password
|
|||||||
S3_FORCE_PATH_STYLE=true
|
S3_FORCE_PATH_STYLE=true
|
||||||
|
|
||||||
# Future local services
|
# Future local services
|
||||||
PUBLIC_API_BASE_URL=http://localhost:3001
|
BACKEND_PORT=3001
|
||||||
|
ADMIN_PORT=5173
|
||||||
|
GATEWAY_HOST=0.0.0.0
|
||||||
|
GATEWAY_PORT=8888
|
||||||
|
PUBLIC_BACKEND_BASE_URL=http://localhost:3001
|
||||||
|
PUBLIC_ADMIN_BASE_URL=http://localhost:5173
|
||||||
PUBLIC_IMAGE_BASE_URL=http://localhost:8888
|
PUBLIC_IMAGE_BASE_URL=http://localhost:8888
|
||||||
|
|
||||||
# imgproxy is always external for image-platform.
|
# Gateway proxies /api and Swagger routes to this upstream.
|
||||||
# Local example: run imgproxy separately on localhost:18080.
|
GATEWAY_BACKEND_UPSTREAM=http://localhost:3001
|
||||||
|
|
||||||
|
# Dev imgproxy is exposed only on localhost.
|
||||||
|
IMGPROXY_PORT=18080
|
||||||
IMGPROXY_UPSTREAM=http://localhost:18080
|
IMGPROXY_UPSTREAM=http://localhost:18080
|
||||||
IMGPROXY_SIGNING_ENABLED=false
|
IMGPROXY_SIGNING_ENABLED=false
|
||||||
IMGPROXY_KEY=
|
IMGPROXY_KEY=
|
||||||
IMGPROXY_SALT=
|
IMGPROXY_SALT=
|
||||||
|
IMGPROXY_WORKERS=2
|
||||||
|
IMGPROXY_MAX_SRC_RESOLUTION=20
|
||||||
|
IMGPROXY_DOWNLOAD_TIMEOUT=30
|
||||||
|
IMGPROXY_ALLOWED_SOURCES=
|
||||||
|
|
||||||
|
# RabbitMQ dev broker is exposed only on localhost.
|
||||||
|
RABBITMQ_DEFAULT_USER=image
|
||||||
|
RABBITMQ_DEFAULT_PASS=image-password
|
||||||
|
RABBITMQ_DEFAULT_VHOST=image_platform
|
||||||
|
RABBITMQ_PORT=5672
|
||||||
|
RABBITMQ_MANAGEMENT_PORT=15672
|
||||||
|
RABBITMQ_URL=amqp://image:image-password@localhost:5672/image_platform
|
||||||
|
WORKER_PREFETCH=2
|
||||||
|
|
||||||
|
# Queue topology
|
||||||
|
RABBITMQ_JOBS_EXCHANGE=image-platform.jobs
|
||||||
|
RABBITMQ_GENERATE_VARIANT_QUEUE=image.generate-variant
|
||||||
|
RABBITMQ_GENERATE_VARIANT_DLX=image-platform.jobs.dlx
|
||||||
|
RABBITMQ_GENERATE_VARIANT_DLQ=image.generate-variant.dlq
|
||||||
|
|||||||
38
README.md
38
README.md
@@ -4,16 +4,18 @@ Image Platform - отдельная площадка для управления
|
|||||||
|
|
||||||
## Статус
|
## Статус
|
||||||
|
|
||||||
Сейчас создан только базовый monorepo и dev-инфраструктура. Приложения `api`, `admin` и `gateway` пока намеренно не созданы.
|
Сейчас создан базовый monorepo, dev-инфраструктура, NestJS backend с Swagger, чистый Vite React TS admin, Fastify gateway skeleton, Drizzle database package, shared queue/storage packages и worker skeleton.
|
||||||
|
|
||||||
## Целевая схема
|
## Целевая схема
|
||||||
|
|
||||||
```text
|
```text
|
||||||
client
|
client
|
||||||
-> CDN optional
|
-> CDN optional
|
||||||
-> gateway Caddy/Souin hot cache
|
-> Fastify gateway + L1 memory cache
|
||||||
|
-> NestJS backend
|
||||||
-> S3/Object Storage persistent variants
|
-> S3/Object Storage persistent variants
|
||||||
-> generator/worker
|
-> generator/worker on miss
|
||||||
|
-> RabbitMQ
|
||||||
-> external imgproxy
|
-> external imgproxy
|
||||||
-> source/original image
|
-> source/original image
|
||||||
```
|
```
|
||||||
@@ -27,20 +29,28 @@ client
|
|||||||
- PostgreSQL
|
- PostgreSQL
|
||||||
- MinIO
|
- MinIO
|
||||||
- MinIO bucket init
|
- MinIO bucket init
|
||||||
|
- imgproxy dev instance
|
||||||
|
- RabbitMQ
|
||||||
|
|
||||||
Позже нодой будут запускаться:
|
Нодой запускается:
|
||||||
|
|
||||||
- NestJS API
|
- NestJS backend
|
||||||
- worker
|
|
||||||
- React/Vite admin
|
- React/Vite admin
|
||||||
|
- Fastify gateway
|
||||||
|
- worker
|
||||||
|
|
||||||
Gateway будет добавлен отдельно позже.
|
Gateway уже добавлен как JS/Fastify skeleton. Сейчас `/images/*` возвращает `501`, пока не подключены DB/S3/imgproxy.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cp .env.example .env
|
cp .env.example .env
|
||||||
pnpm install
|
pnpm install
|
||||||
pnpm infra:up
|
pnpm infra:up
|
||||||
|
pnpm db:migrate
|
||||||
pnpm infra:config
|
pnpm infra:config
|
||||||
|
pnpm backend:dev
|
||||||
|
pnpm admin:dev
|
||||||
|
pnpm gateway:dev
|
||||||
|
pnpm worker:dev
|
||||||
```
|
```
|
||||||
|
|
||||||
Порты по умолчанию:
|
Порты по умолчанию:
|
||||||
@@ -50,11 +60,21 @@ pnpm infra:config
|
|||||||
| PostgreSQL | `localhost:5433` |
|
| PostgreSQL | `localhost:5433` |
|
||||||
| MinIO API | `http://localhost:9000` |
|
| MinIO API | `http://localhost:9000` |
|
||||||
| MinIO Console | `http://localhost:9001` |
|
| MinIO Console | `http://localhost:9001` |
|
||||||
|
| imgproxy | `http://localhost:18080` |
|
||||||
|
| RabbitMQ | `amqp://localhost:5672` |
|
||||||
|
| RabbitMQ Management | `http://localhost:15672` |
|
||||||
|
| Backend API | `http://localhost:3001/api` |
|
||||||
|
| Swagger | `http://localhost:3001/docs` |
|
||||||
|
| Admin | `http://localhost:5173` |
|
||||||
|
| Gateway | `http://localhost:8888` |
|
||||||
|
|
||||||
|
Если старый локальный `image-gateway` уже занимает `8888`, остановите его или задайте другой `GATEWAY_PORT` в `.env`.
|
||||||
|
|
||||||
## Документация
|
## Документация
|
||||||
|
|
||||||
- `docs/architecture.md` - целевая архитектура и ответственность компонентов.
|
- `docs/architecture.md` - целевая архитектура и ответственность компонентов.
|
||||||
- `docs/development.md` - локальный dev flow.
|
- `docs/development.md` - локальный dev flow.
|
||||||
- `docs/data-model.md` - черновик PostgreSQL модели.
|
- `docs/data-model.md` - текущая Drizzle/PostgreSQL модель.
|
||||||
- `docs/api-contract-draft.md` - черновик будущего JSON API.
|
- `docs/backend-contract-draft.md` - черновик будущего backend-контракта.
|
||||||
- `docs/imgproxy-contract.md` - контракт с external imgproxy.
|
- `docs/imgproxy-contract.md` - контракт с external imgproxy.
|
||||||
|
- `docs/next-image-provider.md` - контракт custom provider для `next/image`.
|
||||||
|
|||||||
12
apps/admin/index.html
Normal file
12
apps/admin/index.html
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
<!doctype html>
|
||||||
|
<html lang="ru">
|
||||||
|
<head>
|
||||||
|
<meta charset="UTF-8" />
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||||
|
<title>Image Platform Admin</title>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<div id="root"></div>
|
||||||
|
<script type="module" src="/src/main.tsx"></script>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
24
apps/admin/package.json
Normal file
24
apps/admin/package.json
Normal file
@@ -0,0 +1,24 @@
|
|||||||
|
{
|
||||||
|
"name": "@image-platform/admin",
|
||||||
|
"version": "0.1.0",
|
||||||
|
"private": true,
|
||||||
|
"type": "module",
|
||||||
|
"scripts": {
|
||||||
|
"build": "tsc -b && vite build",
|
||||||
|
"dev": "vite --host 0.0.0.0 --port 5173",
|
||||||
|
"preview": "vite preview --host 0.0.0.0 --port 5173",
|
||||||
|
"typecheck": "tsc -b"
|
||||||
|
},
|
||||||
|
"dependencies": {
|
||||||
|
"react": "^19.2.5",
|
||||||
|
"react-dom": "^19.2.5"
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"@types/node": "^24.12.2",
|
||||||
|
"@types/react": "^19.2.14",
|
||||||
|
"@types/react-dom": "^19.2.3",
|
||||||
|
"@vitejs/plugin-react": "^6.0.1",
|
||||||
|
"typescript": "^5.9.3",
|
||||||
|
"vite": "^8.0.10"
|
||||||
|
}
|
||||||
|
}
|
||||||
111
apps/admin/src/App.css
Normal file
111
apps/admin/src/App.css
Normal file
@@ -0,0 +1,111 @@
|
|||||||
|
:root {
|
||||||
|
color: #171411;
|
||||||
|
background: #f7f4ee;
|
||||||
|
font-family: Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
|
||||||
|
font-synthesis: none;
|
||||||
|
text-rendering: optimizeLegibility;
|
||||||
|
-webkit-font-smoothing: antialiased;
|
||||||
|
-moz-osx-font-smoothing: grayscale;
|
||||||
|
}
|
||||||
|
|
||||||
|
* {
|
||||||
|
box-sizing: border-box;
|
||||||
|
}
|
||||||
|
|
||||||
|
body {
|
||||||
|
min-width: 320px;
|
||||||
|
min-height: 100vh;
|
||||||
|
margin: 0;
|
||||||
|
background:
|
||||||
|
radial-gradient(circle at 18% 18%, rgba(123, 76, 255, 0.18), transparent 30rem),
|
||||||
|
#f7f4ee;
|
||||||
|
}
|
||||||
|
|
||||||
|
button,
|
||||||
|
input,
|
||||||
|
textarea,
|
||||||
|
select {
|
||||||
|
font: inherit;
|
||||||
|
}
|
||||||
|
|
||||||
|
.app-shell {
|
||||||
|
min-height: 100vh;
|
||||||
|
padding: 48px 24px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.hero {
|
||||||
|
max-width: 960px;
|
||||||
|
margin: 0 auto;
|
||||||
|
padding: 40px;
|
||||||
|
border: 1px solid #e4ded4;
|
||||||
|
border-radius: 32px;
|
||||||
|
background: rgba(255, 255, 255, 0.82);
|
||||||
|
box-shadow: 0 22px 80px rgba(40, 32, 21, 0.08);
|
||||||
|
}
|
||||||
|
|
||||||
|
.eyebrow {
|
||||||
|
margin: 0 0 16px;
|
||||||
|
color: #7b4cff;
|
||||||
|
font-size: 13px;
|
||||||
|
font-weight: 700;
|
||||||
|
letter-spacing: 0.22em;
|
||||||
|
text-transform: uppercase;
|
||||||
|
}
|
||||||
|
|
||||||
|
h1 {
|
||||||
|
max-width: 760px;
|
||||||
|
margin: 0;
|
||||||
|
font-size: clamp(40px, 7vw, 76px);
|
||||||
|
line-height: 0.92;
|
||||||
|
letter-spacing: -0.06em;
|
||||||
|
}
|
||||||
|
|
||||||
|
.lead {
|
||||||
|
max-width: 680px;
|
||||||
|
margin: 24px 0 0;
|
||||||
|
color: #73695d;
|
||||||
|
font-size: 18px;
|
||||||
|
line-height: 1.7;
|
||||||
|
}
|
||||||
|
|
||||||
|
.cards {
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: repeat(3, minmax(0, 1fr));
|
||||||
|
gap: 16px;
|
||||||
|
max-width: 960px;
|
||||||
|
margin: 24px auto 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.card {
|
||||||
|
min-height: 150px;
|
||||||
|
padding: 24px;
|
||||||
|
border: 1px solid #e4ded4;
|
||||||
|
border-radius: 24px;
|
||||||
|
background: rgba(255, 255, 255, 0.76);
|
||||||
|
}
|
||||||
|
|
||||||
|
.card h2 {
|
||||||
|
margin: 0;
|
||||||
|
font-size: 18px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.card p {
|
||||||
|
margin: 12px 0 0;
|
||||||
|
color: #73695d;
|
||||||
|
line-height: 1.55;
|
||||||
|
}
|
||||||
|
|
||||||
|
@media (max-width: 720px) {
|
||||||
|
.app-shell {
|
||||||
|
padding: 24px 16px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.hero {
|
||||||
|
padding: 28px;
|
||||||
|
border-radius: 24px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.cards {
|
||||||
|
grid-template-columns: 1fr;
|
||||||
|
}
|
||||||
|
}
|
||||||
40
apps/admin/src/App.tsx
Normal file
40
apps/admin/src/App.tsx
Normal file
@@ -0,0 +1,40 @@
|
|||||||
|
import "./App.css"
|
||||||
|
|
||||||
|
const cards = [
|
||||||
|
{
|
||||||
|
title: "Assets",
|
||||||
|
description: "Каталог исходных изображений и связанной metadata.",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
title: "Variants",
|
||||||
|
description: "Будущие AVIF/WebP/JPEG variants, presets и статусы генерации.",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
title: "Storage",
|
||||||
|
description: "PostgreSQL как source of truth, S3/MinIO как хранилище bytes.",
|
||||||
|
},
|
||||||
|
]
|
||||||
|
|
||||||
|
export function App() {
|
||||||
|
return (
|
||||||
|
<main className="app-shell">
|
||||||
|
<section className="hero">
|
||||||
|
<p className="eyebrow">Image Platform Admin</p>
|
||||||
|
<h1>чистый Vite React TS app</h1>
|
||||||
|
<p className="lead">
|
||||||
|
Это стартовая админка без UI-фреймворков. Дальше сюда добавим управление allowed hosts,
|
||||||
|
assets, variants и presets.
|
||||||
|
</p>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<section className="cards" aria-label="Будущие разделы">
|
||||||
|
{cards.map((card) => (
|
||||||
|
<article className="card" key={card.title}>
|
||||||
|
<h2>{card.title}</h2>
|
||||||
|
<p>{card.description}</p>
|
||||||
|
</article>
|
||||||
|
))}
|
||||||
|
</section>
|
||||||
|
</main>
|
||||||
|
)
|
||||||
|
}
|
||||||
10
apps/admin/src/main.tsx
Normal file
10
apps/admin/src/main.tsx
Normal file
@@ -0,0 +1,10 @@
|
|||||||
|
import { StrictMode } from "react"
|
||||||
|
import { createRoot } from "react-dom/client"
|
||||||
|
|
||||||
|
import { App } from "./App"
|
||||||
|
|
||||||
|
createRoot(document.getElementById("root")!).render(
|
||||||
|
<StrictMode>
|
||||||
|
<App />
|
||||||
|
</StrictMode>,
|
||||||
|
)
|
||||||
22
apps/admin/tsconfig.app.json
Normal file
22
apps/admin/tsconfig.app.json
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
{
|
||||||
|
"compilerOptions": {
|
||||||
|
"composite": true,
|
||||||
|
"tsBuildInfoFile": "./node_modules/.tmp/tsconfig.app.tsbuildinfo",
|
||||||
|
"target": "ES2022",
|
||||||
|
"useDefineForClassFields": true,
|
||||||
|
"lib": ["ES2022", "DOM", "DOM.Iterable"],
|
||||||
|
"allowJs": false,
|
||||||
|
"skipLibCheck": true,
|
||||||
|
"esModuleInterop": true,
|
||||||
|
"allowSyntheticDefaultImports": true,
|
||||||
|
"strict": true,
|
||||||
|
"forceConsistentCasingInFileNames": true,
|
||||||
|
"module": "ESNext",
|
||||||
|
"moduleResolution": "Bundler",
|
||||||
|
"resolveJsonModule": true,
|
||||||
|
"isolatedModules": true,
|
||||||
|
"noEmit": true,
|
||||||
|
"jsx": "react-jsx"
|
||||||
|
},
|
||||||
|
"include": ["src"]
|
||||||
|
}
|
||||||
11
apps/admin/tsconfig.json
Normal file
11
apps/admin/tsconfig.json
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
{
|
||||||
|
"files": [],
|
||||||
|
"references": [
|
||||||
|
{
|
||||||
|
"path": "./tsconfig.app.json"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"path": "./tsconfig.node.json"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
18
apps/admin/tsconfig.node.json
Normal file
18
apps/admin/tsconfig.node.json
Normal file
@@ -0,0 +1,18 @@
|
|||||||
|
{
|
||||||
|
"compilerOptions": {
|
||||||
|
"composite": true,
|
||||||
|
"tsBuildInfoFile": "./node_modules/.tmp/tsconfig.node.tsbuildinfo",
|
||||||
|
"target": "ES2023",
|
||||||
|
"lib": ["ES2023"],
|
||||||
|
"allowJs": false,
|
||||||
|
"skipLibCheck": true,
|
||||||
|
"strict": true,
|
||||||
|
"module": "ESNext",
|
||||||
|
"moduleResolution": "Bundler",
|
||||||
|
"allowSyntheticDefaultImports": true,
|
||||||
|
"isolatedModules": true,
|
||||||
|
"noEmit": true,
|
||||||
|
"types": ["node"]
|
||||||
|
},
|
||||||
|
"include": ["vite.config.ts"]
|
||||||
|
}
|
||||||
6
apps/admin/vite.config.ts
Normal file
6
apps/admin/vite.config.ts
Normal file
@@ -0,0 +1,6 @@
|
|||||||
|
import react from "@vitejs/plugin-react"
|
||||||
|
import { defineConfig } from "vite"
|
||||||
|
|
||||||
|
export default defineConfig({
|
||||||
|
plugins: [react()],
|
||||||
|
})
|
||||||
8
apps/backend/nest-cli.json
Normal file
8
apps/backend/nest-cli.json
Normal file
@@ -0,0 +1,8 @@
|
|||||||
|
{
|
||||||
|
"$schema": "https://json.schemastore.org/nest-cli",
|
||||||
|
"collection": "@nestjs/schematics",
|
||||||
|
"sourceRoot": "src",
|
||||||
|
"compilerOptions": {
|
||||||
|
"deleteOutDir": true
|
||||||
|
}
|
||||||
|
}
|
||||||
27
apps/backend/package.json
Normal file
27
apps/backend/package.json
Normal file
@@ -0,0 +1,27 @@
|
|||||||
|
{
|
||||||
|
"name": "@image-platform/backend",
|
||||||
|
"version": "0.1.0",
|
||||||
|
"private": true,
|
||||||
|
"scripts": {
|
||||||
|
"build": "nest build",
|
||||||
|
"dev": "nest start --watch",
|
||||||
|
"start": "node dist/main.js",
|
||||||
|
"typecheck": "tsc --noEmit -p tsconfig.json"
|
||||||
|
},
|
||||||
|
"dependencies": {
|
||||||
|
"@nestjs/common": "^11.0.0",
|
||||||
|
"@nestjs/core": "^11.0.0",
|
||||||
|
"@nestjs/platform-express": "^11.0.0",
|
||||||
|
"@nestjs/swagger": "^11.0.0",
|
||||||
|
"reflect-metadata": "^0.2.2",
|
||||||
|
"rxjs": "^7.8.1",
|
||||||
|
"swagger-ui-express": "^5.0.1"
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"@nestjs/cli": "^11.0.0",
|
||||||
|
"@nestjs/schematics": "^11.0.0",
|
||||||
|
"@types/node": "^24.0.0",
|
||||||
|
"@types/swagger-ui-express": "^4.1.8",
|
||||||
|
"typescript": "^5.9.0"
|
||||||
|
}
|
||||||
|
}
|
||||||
9
apps/backend/src/app.module.ts
Normal file
9
apps/backend/src/app.module.ts
Normal file
@@ -0,0 +1,9 @@
|
|||||||
|
import { Module } from "@nestjs/common"
|
||||||
|
|
||||||
|
import { HealthController } from "./health/health.controller"
|
||||||
|
import { InternalImagesController } from "./internal-images/internal-images.controller"
|
||||||
|
|
||||||
|
@Module({
|
||||||
|
controllers: [HealthController, InternalImagesController],
|
||||||
|
})
|
||||||
|
export class AppModule {}
|
||||||
9
apps/backend/src/health/health-response.dto.ts
Normal file
9
apps/backend/src/health/health-response.dto.ts
Normal file
@@ -0,0 +1,9 @@
|
|||||||
|
import { ApiProperty } from "@nestjs/swagger"
|
||||||
|
|
||||||
|
export class HealthResponseDto {
|
||||||
|
@ApiProperty({ example: "image-platform-api" })
|
||||||
|
service!: string
|
||||||
|
|
||||||
|
@ApiProperty({ example: "ok" })
|
||||||
|
status!: string
|
||||||
|
}
|
||||||
18
apps/backend/src/health/health.controller.ts
Normal file
18
apps/backend/src/health/health.controller.ts
Normal file
@@ -0,0 +1,18 @@
|
|||||||
|
import { Controller, Get } from "@nestjs/common"
|
||||||
|
import { ApiOkResponse, ApiOperation, ApiTags } from "@nestjs/swagger"
|
||||||
|
|
||||||
|
import { HealthResponseDto } from "./health-response.dto"
|
||||||
|
|
||||||
|
@ApiTags("system")
|
||||||
|
@Controller("health")
|
||||||
|
export class HealthController {
|
||||||
|
@Get()
|
||||||
|
@ApiOperation({ summary: "Проверить состояние API" })
|
||||||
|
@ApiOkResponse({ type: HealthResponseDto })
|
||||||
|
getHealth(): HealthResponseDto {
|
||||||
|
return {
|
||||||
|
service: "image-platform-api",
|
||||||
|
status: "ok",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
21
apps/backend/src/internal-images/ensure-image-variant.dto.ts
Normal file
21
apps/backend/src/internal-images/ensure-image-variant.dto.ts
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
import { ApiProperty } from "@nestjs/swagger"
|
||||||
|
|
||||||
|
export class EnsureImageVariantRequestDto {
|
||||||
|
@ApiProperty({ example: "asset_123" })
|
||||||
|
assetId!: string
|
||||||
|
|
||||||
|
@ApiProperty({ example: 4, minimum: 1 })
|
||||||
|
version!: number
|
||||||
|
|
||||||
|
@ApiProperty({ example: "card" })
|
||||||
|
preset!: string
|
||||||
|
|
||||||
|
@ApiProperty({ example: 640, minimum: 1 })
|
||||||
|
width!: number
|
||||||
|
|
||||||
|
@ApiProperty({ example: 80, minimum: 1 })
|
||||||
|
quality!: number
|
||||||
|
|
||||||
|
@ApiProperty({ enum: ["auto", "avif", "webp", "jpg", "png"], example: "auto" })
|
||||||
|
format!: "auto" | "avif" | "jpg" | "png" | "webp"
|
||||||
|
}
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
import { Body, Controller, NotImplementedException, Post } from "@nestjs/common"
|
||||||
|
import { ApiOperation, ApiResponse, ApiTags } from "@nestjs/swagger"
|
||||||
|
|
||||||
|
import { EnsureImageVariantRequestDto } from "./ensure-image-variant.dto"
|
||||||
|
|
||||||
|
@ApiTags("internal-images")
|
||||||
|
@Controller("internal/images")
|
||||||
|
export class InternalImagesController {
|
||||||
|
@Post("ensure")
|
||||||
|
@ApiOperation({ summary: "Ensure image variant for Gateway L1 miss" })
|
||||||
|
@ApiResponse({ status: 501, description: "Read-through image pipeline is not implemented yet" })
|
||||||
|
ensureImageVariant(@Body() request: EnsureImageVariantRequestDto): never {
|
||||||
|
throw new NotImplementedException({
|
||||||
|
message: "image read-through pipeline is not implemented yet",
|
||||||
|
request,
|
||||||
|
status: "not_implemented",
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
37
apps/backend/src/main.ts
Normal file
37
apps/backend/src/main.ts
Normal file
@@ -0,0 +1,37 @@
|
|||||||
|
import { NestFactory } from "@nestjs/core"
|
||||||
|
import { DocumentBuilder, SwaggerModule } from "@nestjs/swagger"
|
||||||
|
|
||||||
|
import { AppModule } from "./app.module"
|
||||||
|
|
||||||
|
async function bootstrap() {
|
||||||
|
const app = await NestFactory.create(AppModule)
|
||||||
|
|
||||||
|
app.setGlobalPrefix("api")
|
||||||
|
app.enableShutdownHooks()
|
||||||
|
|
||||||
|
const openApiConfig = new DocumentBuilder()
|
||||||
|
.setTitle("Image Platform API")
|
||||||
|
.setDescription("Control plane for image assets, variants, S3 storage and external imgproxy.")
|
||||||
|
.setVersion("0.1.0")
|
||||||
|
.addTag("system")
|
||||||
|
.addTag("assets")
|
||||||
|
.addTag("variants")
|
||||||
|
.addTag("allowed-hosts")
|
||||||
|
.addTag("internal-images")
|
||||||
|
.build()
|
||||||
|
|
||||||
|
const openApiDocument = SwaggerModule.createDocument(app, openApiConfig)
|
||||||
|
|
||||||
|
SwaggerModule.setup("docs", app, openApiDocument, {
|
||||||
|
jsonDocumentUrl: "docs-json",
|
||||||
|
swaggerOptions: {
|
||||||
|
persistAuthorization: true,
|
||||||
|
},
|
||||||
|
})
|
||||||
|
|
||||||
|
const port = Number.parseInt(process.env.API_PORT ?? "3001", 10)
|
||||||
|
|
||||||
|
await app.listen(port)
|
||||||
|
}
|
||||||
|
|
||||||
|
void bootstrap()
|
||||||
8
apps/backend/tsconfig.build.json
Normal file
8
apps/backend/tsconfig.build.json
Normal file
@@ -0,0 +1,8 @@
|
|||||||
|
{
|
||||||
|
"extends": "./tsconfig.json",
|
||||||
|
"compilerOptions": {
|
||||||
|
"declaration": true,
|
||||||
|
"noEmit": false
|
||||||
|
},
|
||||||
|
"exclude": ["node_modules", "dist", "test", "**/*.spec.ts"]
|
||||||
|
}
|
||||||
23
apps/backend/tsconfig.json
Normal file
23
apps/backend/tsconfig.json
Normal file
@@ -0,0 +1,23 @@
|
|||||||
|
{
|
||||||
|
"compilerOptions": {
|
||||||
|
"allowSyntheticDefaultImports": true,
|
||||||
|
"baseUrl": "./",
|
||||||
|
"declaration": true,
|
||||||
|
"emitDecoratorMetadata": true,
|
||||||
|
"esModuleInterop": true,
|
||||||
|
"experimentalDecorators": true,
|
||||||
|
"incremental": true,
|
||||||
|
"module": "commonjs",
|
||||||
|
"moduleResolution": "node",
|
||||||
|
"outDir": "./dist",
|
||||||
|
"removeComments": true,
|
||||||
|
"skipLibCheck": true,
|
||||||
|
"sourceMap": true,
|
||||||
|
"strict": true,
|
||||||
|
"strictPropertyInitialization": false,
|
||||||
|
"target": "ES2023",
|
||||||
|
"types": ["node"]
|
||||||
|
},
|
||||||
|
"include": ["src/**/*.ts"],
|
||||||
|
"exclude": ["node_modules", "dist"]
|
||||||
|
}
|
||||||
20
apps/gateway/package.json
Normal file
20
apps/gateway/package.json
Normal file
@@ -0,0 +1,20 @@
|
|||||||
|
{
|
||||||
|
"name": "@image-platform/gateway",
|
||||||
|
"version": "0.1.0",
|
||||||
|
"private": true,
|
||||||
|
"type": "module",
|
||||||
|
"scripts": {
|
||||||
|
"build": "tsc -p tsconfig.build.json",
|
||||||
|
"dev": "tsx watch src/main.ts",
|
||||||
|
"start": "node dist/main.js",
|
||||||
|
"typecheck": "tsc --noEmit -p tsconfig.json"
|
||||||
|
},
|
||||||
|
"dependencies": {
|
||||||
|
"fastify": "^5.8.5"
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"@types/node": "^25.6.0",
|
||||||
|
"tsx": "^4.21.0",
|
||||||
|
"typescript": "^6.0.3"
|
||||||
|
}
|
||||||
|
}
|
||||||
27
apps/gateway/src/config.ts
Normal file
27
apps/gateway/src/config.ts
Normal file
@@ -0,0 +1,27 @@
|
|||||||
|
export type GatewayConfig = {
|
||||||
|
backendUpstream: URL
|
||||||
|
host: string
|
||||||
|
port: number
|
||||||
|
}
|
||||||
|
|
||||||
|
export function loadGatewayConfig(): GatewayConfig {
|
||||||
|
return {
|
||||||
|
backendUpstream: new URL(process.env.GATEWAY_BACKEND_UPSTREAM ?? "http://localhost:3001"),
|
||||||
|
host: process.env.GATEWAY_HOST ?? "0.0.0.0",
|
||||||
|
port: parsePort(process.env.GATEWAY_PORT, 8888),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function parsePort(value: string | undefined, fallback: number) {
|
||||||
|
if (!value) {
|
||||||
|
return fallback
|
||||||
|
}
|
||||||
|
|
||||||
|
const parsed = Number.parseInt(value, 10)
|
||||||
|
|
||||||
|
if (!Number.isFinite(parsed) || parsed < 1 || parsed > 65535) {
|
||||||
|
throw new Error(`Invalid port: ${value}`)
|
||||||
|
}
|
||||||
|
|
||||||
|
return parsed
|
||||||
|
}
|
||||||
12
apps/gateway/src/main.ts
Normal file
12
apps/gateway/src/main.ts
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
import { loadGatewayConfig } from "./config.js"
|
||||||
|
import { createGatewayServer } from "./server.js"
|
||||||
|
|
||||||
|
const config = loadGatewayConfig()
|
||||||
|
const app = createGatewayServer(config)
|
||||||
|
|
||||||
|
try {
|
||||||
|
await app.listen({ host: config.host, port: config.port })
|
||||||
|
} catch (error) {
|
||||||
|
app.log.error(error)
|
||||||
|
process.exit(1)
|
||||||
|
}
|
||||||
77
apps/gateway/src/proxy.ts
Normal file
77
apps/gateway/src/proxy.ts
Normal file
@@ -0,0 +1,77 @@
|
|||||||
|
import { Readable } from "node:stream"
|
||||||
|
|
||||||
|
import type { FastifyReply, FastifyRequest } from "fastify"
|
||||||
|
|
||||||
|
const hopByHopHeaders = new Set([
|
||||||
|
"connection",
|
||||||
|
"keep-alive",
|
||||||
|
"proxy-authenticate",
|
||||||
|
"proxy-authorization",
|
||||||
|
"te",
|
||||||
|
"trailer",
|
||||||
|
"transfer-encoding",
|
||||||
|
"upgrade",
|
||||||
|
])
|
||||||
|
|
||||||
|
export async function proxyToUpstream(request: FastifyRequest, reply: FastifyReply, upstreamBaseUrl: URL) {
|
||||||
|
const upstreamUrl = new URL(request.url, upstreamBaseUrl)
|
||||||
|
const headers = buildProxyHeaders(request)
|
||||||
|
const init: RequestInit & { duplex?: "half" } = {
|
||||||
|
headers,
|
||||||
|
method: request.method,
|
||||||
|
redirect: "manual",
|
||||||
|
}
|
||||||
|
|
||||||
|
if (request.method !== "GET" && request.method !== "HEAD") {
|
||||||
|
init.body = request.raw as RequestInit["body"]
|
||||||
|
init.duplex = "half"
|
||||||
|
}
|
||||||
|
|
||||||
|
const response = await fetch(upstreamUrl, init)
|
||||||
|
|
||||||
|
reply.code(response.status)
|
||||||
|
|
||||||
|
response.headers.forEach((value, key) => {
|
||||||
|
if (!hopByHopHeaders.has(key.toLowerCase())) {
|
||||||
|
reply.header(key, value)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
if (!response.body) {
|
||||||
|
return reply.send()
|
||||||
|
}
|
||||||
|
|
||||||
|
return reply.send(Readable.fromWeb(response.body))
|
||||||
|
}
|
||||||
|
|
||||||
|
function buildProxyHeaders(request: FastifyRequest) {
|
||||||
|
const headers = new Headers()
|
||||||
|
|
||||||
|
for (const [key, rawValue] of Object.entries(request.headers)) {
|
||||||
|
const lowerKey = key.toLowerCase()
|
||||||
|
|
||||||
|
if (lowerKey === "host" || hopByHopHeaders.has(lowerKey)) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
if (Array.isArray(rawValue)) {
|
||||||
|
for (const value of rawValue) {
|
||||||
|
headers.append(key, value)
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
if (typeof rawValue === "string") {
|
||||||
|
headers.set(key, rawValue)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
headers.set("x-forwarded-host", request.headers.host ?? "")
|
||||||
|
headers.set("x-forwarded-proto", "http")
|
||||||
|
|
||||||
|
if (request.ip) {
|
||||||
|
headers.set("x-forwarded-for", request.ip)
|
||||||
|
}
|
||||||
|
|
||||||
|
return headers
|
||||||
|
}
|
||||||
90
apps/gateway/src/server.ts
Normal file
90
apps/gateway/src/server.ts
Normal file
@@ -0,0 +1,90 @@
|
|||||||
|
import Fastify from "fastify"
|
||||||
|
|
||||||
|
import type { GatewayConfig } from "./config.js"
|
||||||
|
import { proxyToUpstream } from "./proxy.js"
|
||||||
|
|
||||||
|
export function createGatewayServer(config: GatewayConfig) {
|
||||||
|
const app = Fastify({
|
||||||
|
logger: true,
|
||||||
|
})
|
||||||
|
|
||||||
|
app.get("/health", async () => ({
|
||||||
|
service: "image-platform-gateway",
|
||||||
|
status: "ok",
|
||||||
|
}))
|
||||||
|
|
||||||
|
app.get<{ Params: ImageParams; Querystring: ImageQuery }>(
|
||||||
|
"/images/:assetId/:version/:preset",
|
||||||
|
async (request, reply) => {
|
||||||
|
const version = parseVersionParam(request.params.version)
|
||||||
|
|
||||||
|
if (version === null) {
|
||||||
|
return reply.code(400).send({
|
||||||
|
message: "image version must use v{number} format",
|
||||||
|
statusCode: 400,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
const width = parseOptionalInteger(request.query.w)
|
||||||
|
const quality = parseOptionalInteger(request.query.q)
|
||||||
|
const format = request.query.f ?? "auto"
|
||||||
|
|
||||||
|
return reply.code(501).header("cache-control", "no-store").send({
|
||||||
|
assetId: request.params.assetId,
|
||||||
|
format,
|
||||||
|
message: "image gateway read-through pipeline is not implemented yet",
|
||||||
|
preset: request.params.preset,
|
||||||
|
quality,
|
||||||
|
status: "not_implemented",
|
||||||
|
version,
|
||||||
|
width,
|
||||||
|
})
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
app.all("/api/*", async (request, reply) => proxyToUpstream(request, reply, config.backendUpstream))
|
||||||
|
app.all("/docs", async (request, reply) => proxyToUpstream(request, reply, config.backendUpstream))
|
||||||
|
app.all("/docs/*", async (request, reply) => proxyToUpstream(request, reply, config.backendUpstream))
|
||||||
|
app.all("/docs-json", async (request, reply) => proxyToUpstream(request, reply, config.backendUpstream))
|
||||||
|
|
||||||
|
app.setNotFoundHandler(async (_request, reply) => {
|
||||||
|
return reply.code(404).send({
|
||||||
|
message: "route not found",
|
||||||
|
statusCode: 404,
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
return app
|
||||||
|
}
|
||||||
|
|
||||||
|
type ImageQuery = {
|
||||||
|
f?: string
|
||||||
|
q?: string
|
||||||
|
w?: string
|
||||||
|
}
|
||||||
|
|
||||||
|
type ImageParams = {
|
||||||
|
assetId: string
|
||||||
|
preset: string
|
||||||
|
version: string
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseVersionParam(value: string) {
|
||||||
|
if (!value.startsWith("v")) {
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
|
||||||
|
const parsed = Number.parseInt(value.slice(1), 10)
|
||||||
|
|
||||||
|
return Number.isSafeInteger(parsed) && parsed > 0 ? parsed : null
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseOptionalInteger(value: string | undefined) {
|
||||||
|
if (!value) {
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
|
||||||
|
const parsed = Number.parseInt(value, 10)
|
||||||
|
|
||||||
|
return Number.isFinite(parsed) ? parsed : null
|
||||||
|
}
|
||||||
6
apps/gateway/tsconfig.build.json
Normal file
6
apps/gateway/tsconfig.build.json
Normal file
@@ -0,0 +1,6 @@
|
|||||||
|
{
|
||||||
|
"extends": "./tsconfig.json",
|
||||||
|
"compilerOptions": {
|
||||||
|
"noEmit": false
|
||||||
|
}
|
||||||
|
}
|
||||||
22
apps/gateway/tsconfig.json
Normal file
22
apps/gateway/tsconfig.json
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
{
|
||||||
|
"compilerOptions": {
|
||||||
|
"allowSyntheticDefaultImports": true,
|
||||||
|
"declaration": true,
|
||||||
|
"esModuleInterop": true,
|
||||||
|
"forceConsistentCasingInFileNames": true,
|
||||||
|
"lib": ["ES2023"],
|
||||||
|
"module": "NodeNext",
|
||||||
|
"moduleResolution": "NodeNext",
|
||||||
|
"noUncheckedIndexedAccess": true,
|
||||||
|
"outDir": "./dist",
|
||||||
|
"rootDir": "./src",
|
||||||
|
"resolveJsonModule": true,
|
||||||
|
"skipLibCheck": true,
|
||||||
|
"sourceMap": true,
|
||||||
|
"strict": true,
|
||||||
|
"target": "ES2023",
|
||||||
|
"types": ["node"]
|
||||||
|
},
|
||||||
|
"include": ["src/**/*.ts"],
|
||||||
|
"exclude": ["dist", "node_modules"]
|
||||||
|
}
|
||||||
24
apps/worker/package.json
Normal file
24
apps/worker/package.json
Normal file
@@ -0,0 +1,24 @@
|
|||||||
|
{
|
||||||
|
"name": "@image-platform/worker",
|
||||||
|
"version": "0.1.0",
|
||||||
|
"private": true,
|
||||||
|
"type": "module",
|
||||||
|
"scripts": {
|
||||||
|
"build": "tsc -p tsconfig.build.json",
|
||||||
|
"dev": "tsx watch src/main.ts",
|
||||||
|
"start": "node dist/main.js",
|
||||||
|
"typecheck": "tsc --noEmit -p tsconfig.json"
|
||||||
|
},
|
||||||
|
"dependencies": {
|
||||||
|
"@image-platform/database": "workspace:*",
|
||||||
|
"@image-platform/queue": "workspace:*",
|
||||||
|
"@image-platform/storage": "workspace:*",
|
||||||
|
"amqplib": "^1.0.4"
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"@types/amqplib": "^0.10.8",
|
||||||
|
"@types/node": "^25.6.0",
|
||||||
|
"tsx": "^4.21.0",
|
||||||
|
"typescript": "^6.0.3"
|
||||||
|
}
|
||||||
|
}
|
||||||
25
apps/worker/src/config.ts
Normal file
25
apps/worker/src/config.ts
Normal file
@@ -0,0 +1,25 @@
|
|||||||
|
import { loadQueueTopologyFromEnv, type QueueTopology } from "@image-platform/queue"
|
||||||
|
|
||||||
|
export type WorkerConfig = {
|
||||||
|
prefetch: number
|
||||||
|
queueTopology: QueueTopology
|
||||||
|
rabbitmqUrl: string
|
||||||
|
}
|
||||||
|
|
||||||
|
export function loadWorkerConfig(env: NodeJS.ProcessEnv = process.env): WorkerConfig {
|
||||||
|
return {
|
||||||
|
prefetch: parsePositiveInteger(env.WORKER_PREFETCH, 2),
|
||||||
|
queueTopology: loadQueueTopologyFromEnv(env),
|
||||||
|
rabbitmqUrl: env.RABBITMQ_URL ?? "amqp://image:image-password@localhost:5672/image_platform",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function parsePositiveInteger(value: string | undefined, fallback: number) {
|
||||||
|
if (!value) {
|
||||||
|
return fallback
|
||||||
|
}
|
||||||
|
|
||||||
|
const parsed = Number.parseInt(value, 10)
|
||||||
|
|
||||||
|
return Number.isSafeInteger(parsed) && parsed > 0 ? parsed : fallback
|
||||||
|
}
|
||||||
67
apps/worker/src/main.ts
Normal file
67
apps/worker/src/main.ts
Normal file
@@ -0,0 +1,67 @@
|
|||||||
|
import amqp, { type Channel, type ConsumeMessage } from "amqplib"
|
||||||
|
import { parseGenerateVariantJobBuffer, type QueueTopology } from "@image-platform/queue"
|
||||||
|
|
||||||
|
import { loadWorkerConfig } from "./config.js"
|
||||||
|
|
||||||
|
async function bootstrap() {
|
||||||
|
const config = loadWorkerConfig()
|
||||||
|
const connection = await amqp.connect(config.rabbitmqUrl)
|
||||||
|
const channel = await connection.createChannel()
|
||||||
|
|
||||||
|
await assertQueueTopology(channel, config.queueTopology)
|
||||||
|
await channel.prefetch(config.prefetch)
|
||||||
|
await channel.consume(
|
||||||
|
config.queueTopology.generateVariantQueue,
|
||||||
|
(message) => void handleGenerateVariantMessage(channel, message),
|
||||||
|
{ noAck: false },
|
||||||
|
)
|
||||||
|
|
||||||
|
console.log(`worker consuming ${config.queueTopology.generateVariantQueue}`)
|
||||||
|
|
||||||
|
const shutdown = async () => {
|
||||||
|
console.log("worker shutting down")
|
||||||
|
await channel.close().catch((error: unknown) => console.error("failed to close RabbitMQ channel", error))
|
||||||
|
await connection.close().catch((error: unknown) => console.error("failed to close RabbitMQ connection", error))
|
||||||
|
process.exit(0)
|
||||||
|
}
|
||||||
|
|
||||||
|
process.once("SIGINT", () => void shutdown())
|
||||||
|
process.once("SIGTERM", () => void shutdown())
|
||||||
|
}
|
||||||
|
|
||||||
|
async function assertQueueTopology(channel: Channel, topology: QueueTopology) {
|
||||||
|
await channel.assertExchange(topology.jobsExchange, "direct", { durable: true })
|
||||||
|
await channel.assertExchange(topology.jobsDeadLetterExchange, "direct", { durable: true })
|
||||||
|
await channel.assertQueue(topology.generateVariantQueue, {
|
||||||
|
deadLetterExchange: topology.jobsDeadLetterExchange,
|
||||||
|
deadLetterRoutingKey: topology.generateVariantDeadLetterRoutingKey,
|
||||||
|
durable: true,
|
||||||
|
})
|
||||||
|
await channel.assertQueue(topology.generateVariantDeadLetterQueue, { durable: true })
|
||||||
|
await channel.bindQueue(topology.generateVariantQueue, topology.jobsExchange, topology.generateVariantRoutingKey)
|
||||||
|
await channel.bindQueue(
|
||||||
|
topology.generateVariantDeadLetterQueue,
|
||||||
|
topology.jobsDeadLetterExchange,
|
||||||
|
topology.generateVariantDeadLetterRoutingKey,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
async function handleGenerateVariantMessage(channel: Channel, message: ConsumeMessage | null) {
|
||||||
|
if (message === null) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
const job = parseGenerateVariantJobBuffer(message.content)
|
||||||
|
console.log("generate variant job received, handler not implemented yet", job)
|
||||||
|
channel.nack(message, false, false)
|
||||||
|
} catch (error) {
|
||||||
|
console.error("invalid generate variant job", error)
|
||||||
|
channel.nack(message, false, false)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void bootstrap().catch((error: unknown) => {
|
||||||
|
console.error("worker failed to start", error)
|
||||||
|
process.exit(1)
|
||||||
|
})
|
||||||
7
apps/worker/tsconfig.build.json
Normal file
7
apps/worker/tsconfig.build.json
Normal file
@@ -0,0 +1,7 @@
|
|||||||
|
{
|
||||||
|
"extends": "./tsconfig.json",
|
||||||
|
"compilerOptions": {
|
||||||
|
"noEmit": false
|
||||||
|
},
|
||||||
|
"exclude": ["dist", "node_modules", "**/*.spec.ts"]
|
||||||
|
}
|
||||||
21
apps/worker/tsconfig.json
Normal file
21
apps/worker/tsconfig.json
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
{
|
||||||
|
"compilerOptions": {
|
||||||
|
"allowSyntheticDefaultImports": true,
|
||||||
|
"declaration": true,
|
||||||
|
"esModuleInterop": true,
|
||||||
|
"forceConsistentCasingInFileNames": true,
|
||||||
|
"lib": ["ES2023"],
|
||||||
|
"module": "NodeNext",
|
||||||
|
"moduleResolution": "NodeNext",
|
||||||
|
"noUncheckedIndexedAccess": true,
|
||||||
|
"outDir": "./dist",
|
||||||
|
"rootDir": "./src",
|
||||||
|
"skipLibCheck": true,
|
||||||
|
"sourceMap": true,
|
||||||
|
"strict": true,
|
||||||
|
"target": "ES2023",
|
||||||
|
"types": ["node"]
|
||||||
|
},
|
||||||
|
"include": ["src/**/*.ts"],
|
||||||
|
"exclude": ["dist", "node_modules"]
|
||||||
|
}
|
||||||
@@ -1,124 +0,0 @@
|
|||||||
# Черновик API Контракта
|
|
||||||
|
|
||||||
Это не реализация API, а фиксация будущего контракта для NestJS backend.
|
|
||||||
|
|
||||||
Backend отдаёт JSON, metadata, statuses и URLs. Он не должен проксировать image bytes на каждый обычный запрос.
|
|
||||||
|
|
||||||
## Allowed Hosts
|
|
||||||
|
|
||||||
```text
|
|
||||||
GET /allowed-hosts
|
|
||||||
POST /allowed-hosts
|
|
||||||
PATCH /allowed-hosts/:id
|
|
||||||
DELETE /allowed-hosts/:id
|
|
||||||
```
|
|
||||||
|
|
||||||
## Assets
|
|
||||||
|
|
||||||
```text
|
|
||||||
GET /assets
|
|
||||||
POST /assets
|
|
||||||
GET /assets/:id
|
|
||||||
DELETE /assets/:id
|
|
||||||
```
|
|
||||||
|
|
||||||
`POST /assets` request:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"sourceUrl": "https://example.com/photo.jpg"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Responsibilities:
|
|
||||||
|
|
||||||
- validate source URL;
|
|
||||||
- check `allowed_image_hosts`;
|
|
||||||
- create or reuse `image_assets` row;
|
|
||||||
- optionally save original to S3 later.
|
|
||||||
|
|
||||||
## Variants
|
|
||||||
|
|
||||||
```text
|
|
||||||
GET /assets/:id/variants
|
|
||||||
POST /assets/:id/variants
|
|
||||||
POST /variants/:id/regenerate
|
|
||||||
DELETE /variants/:id
|
|
||||||
```
|
|
||||||
|
|
||||||
`POST /assets/:id/variants` request:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"preset": "card",
|
|
||||||
"format": "webp",
|
|
||||||
"width": 640
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Response if ready:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"id": "variant_123",
|
|
||||||
"status": "ready",
|
|
||||||
"url": "http://localhost:8888/images/asset_123/w640_q80_card.webp"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Response if generation is async:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"id": "variant_123",
|
|
||||||
"status": "pending",
|
|
||||||
"url": null
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Image URLs For UI
|
|
||||||
|
|
||||||
Для UI нужен endpoint, который возвращает готовый набор URLs для `<picture>`/`srcset`.
|
|
||||||
|
|
||||||
```text
|
|
||||||
GET /assets/:id/picture?preset=card
|
|
||||||
```
|
|
||||||
|
|
||||||
Example response:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"assetId": "asset_123",
|
|
||||||
"preset": "card",
|
|
||||||
"sources": [
|
|
||||||
{
|
|
||||||
"type": "image/avif",
|
|
||||||
"srcset": "http://localhost:8888/images/asset_123/w320_card.avif 320w, http://localhost:8888/images/asset_123/w640_card.avif 640w"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"type": "image/webp",
|
|
||||||
"srcset": "http://localhost:8888/images/asset_123/w320_card.webp 320w, http://localhost:8888/images/asset_123/w640_card.webp 640w"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"fallback": {
|
|
||||||
"src": "http://localhost:8888/images/asset_123/w640_card.jpg",
|
|
||||||
"width": 640,
|
|
||||||
"height": null
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Worker Lifecycle
|
|
||||||
|
|
||||||
Первый MVP может генерировать sync на request. Если генерация тяжёлая, variant создаётся как `pending`, а worker обрабатывает job.
|
|
||||||
|
|
||||||
PostgreSQL может выступить первой очередью:
|
|
||||||
|
|
||||||
```text
|
|
||||||
SELECT * FROM image_variants
|
|
||||||
WHERE status = 'pending'
|
|
||||||
FOR UPDATE SKIP LOCKED
|
|
||||||
LIMIT 1
|
|
||||||
```
|
|
||||||
|
|
||||||
Позже можно добавить Redis/Valkey или отдельную queue, если PostgreSQL станет узким местом.
|
|
||||||
@@ -12,24 +12,49 @@
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| PostgreSQL | сейчас | Источник правды для assets, variants, hosts, statuses |
|
| PostgreSQL | сейчас | Источник правды для assets, variants, hosts, statuses |
|
||||||
| S3/MinIO | сейчас | Хранилище originals и generated variants |
|
| S3/MinIO | сейчас | Хранилище originals и generated variants |
|
||||||
| API | позже | JSON API, admin operations, validation, orchestration |
|
| Backend | сейчас | NestJS JSON API, Swagger, PostgreSQL/S3/RabbitMQ orchestration |
|
||||||
| Worker | позже | Генерация variants, upload в S3, update PostgreSQL |
|
| Worker | позже | RabbitMQ consumer, imgproxy processing, upload в S3, update PostgreSQL |
|
||||||
| Admin UI | позже | Управление hosts/assets/variants/presets |
|
| Admin UI | сейчас | React/Vite UI для будущего управления hosts/assets/variants/presets |
|
||||||
| Gateway | позже | Caddy/Souin hot cache и delivery layer |
|
| Gateway | сейчас | Fastify public image origin, L1 memory cache, root routing, без DB/S3 доступа |
|
||||||
|
| RabbitMQ | сейчас | Очередь задач генерации variants |
|
||||||
| imgproxy | external | CPU-heavy image processing |
|
| imgproxy | external | CPU-heavy image processing |
|
||||||
|
|
||||||
|
## Архитектурное решение
|
||||||
|
|
||||||
|
Нужное поведение - Cloudinary-like: публичный URL изображения сам запускает read-through pipeline, если variant ещё не создан.
|
||||||
|
|
||||||
|
`image-platform` строится ради совместимости с `next/image` как custom loader provider. Next.js application не должен заранее вызывать API, ждать генерацию и затем подставлять S3 URL. Он должен передавать `src`, `width` и `quality` в loader, а loader должен вернуть стабильный URL нашего image origin.
|
||||||
|
|
||||||
|
Gateway поэтому является обязательной частью public delivery path, а не опциональным кешем.
|
||||||
|
|
||||||
## Целевая delivery схема
|
## Целевая delivery схема
|
||||||
|
|
||||||
```text
|
```text
|
||||||
client
|
client
|
||||||
-> CDN optional
|
-> CDN optional
|
||||||
-> gateway Caddy/Souin
|
-> Fastify gateway L1 memory cache
|
||||||
-> S3 ready variant
|
-> NestJS backend
|
||||||
-> generator fallback
|
-> PostgreSQL + S3 ready variant
|
||||||
|
-> RabbitMQ -> worker
|
||||||
-> external imgproxy
|
-> external imgproxy
|
||||||
-> source image
|
-> source image
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Read-through flow:
|
||||||
|
|
||||||
|
```text
|
||||||
|
1. client запрашивает /images/{assetId}/v{version}/{preset}?w=640&q=80&f=auto
|
||||||
|
2. CDN HIT -> ответ сразу
|
||||||
|
3. Gateway L1 HIT -> ответ сразу
|
||||||
|
4. Gateway L1 MISS -> Gateway вызывает Backend internal ensure endpoint
|
||||||
|
5. Backend проверяет PostgreSQL/S3
|
||||||
|
6. S3 HIT -> Backend стримит bytes Gateway, Gateway кладёт ответ в L1
|
||||||
|
7. S3 MISS -> Backend ставит RabbitMQ job
|
||||||
|
8. Worker вызывает external imgproxy и сохраняет результат в S3
|
||||||
|
9. Worker обновляет PostgreSQL, Backend отдаёт готовые bytes Gateway
|
||||||
|
10. Gateway кладёт ответ в L1 и отдаёт клиенту
|
||||||
|
```
|
||||||
|
|
||||||
## Разделение ответственности
|
## Разделение ответственности
|
||||||
|
|
||||||
PostgreSQL отвечает на вопросы:
|
PostgreSQL отвечает на вопросы:
|
||||||
@@ -49,29 +74,45 @@ S3 хранит байты:
|
|||||||
|
|
||||||
Gateway отдаёт картинки:
|
Gateway отдаёт картинки:
|
||||||
|
|
||||||
- hot cache HIT - сразу из Souin;
|
- L1 memory HIT - сразу из памяти;
|
||||||
- cache MISS - из S3;
|
- L1 memory MISS - вызывает Backend;
|
||||||
- S3 MISS - через generator fallback.
|
- не имеет доступа к PostgreSQL, S3 и RabbitMQ.
|
||||||
|
|
||||||
Backend не должен проксировать картинки на каждый обычный запрос. Он отдаёт JSON, статусы и URLs.
|
Backend управляет процессами: PostgreSQL, S3 read, RabbitMQ enqueue, ожидание ready variant. Worker выполняет CPU-heavy generation через external imgproxy и пишет результат в S3.
|
||||||
|
|
||||||
## URL модель
|
## URL модель
|
||||||
|
|
||||||
Публичные URL должны быть стабильными и не раскрывать source URL:
|
Публичные URL должны быть стабильными и не раскрывать source URL. Для Next/image provider основной URL должен принимать width/quality из loader:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
/images/{assetId}/{variantHash}.{format}
|
/images/{assetId}/v{version}/{preset}?w={width}&q={quality}&f=auto
|
||||||
```
|
```
|
||||||
|
|
||||||
Примеры:
|
Примеры:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
/images/asset_123/w640_q80_cfill.avif
|
/images/asset_123/v4/card?w=640&q=80&f=auto
|
||||||
/images/asset_123/w640_q80_cfill.webp
|
/images/asset_123/v4/hero?w=1920&q=80&f=auto
|
||||||
/images/asset_123/w640_q80_cfill.jpg
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Формат лучше делать явным в URL и отдавать через `<picture>`/`srcset`, а не выбирать по `Accept` header. Так CDN/S3/Gateway cache остаётся предсказуемым.
|
`v{version}` берётся из `image_assets.current_version` и меняется при обновлении source image. Это даёт immutable cache без purge старых CDN/L1/S3 keys.
|
||||||
|
|
||||||
|
`f=auto` нужен для совместимости с `next/image` custom loader: Next передаёт в loader `src`, `width` и `quality`, но не выбирает AVIF/WebP сам при custom loader. Image origin должен выбрать формат по `Accept` header, как Cloudinary `f_auto`.
|
||||||
|
|
||||||
|
Из-за `f=auto` обязательно:
|
||||||
|
|
||||||
|
- S3 key должен включать фактически выбранный формат;
|
||||||
|
- response должен выставлять `Vary: Accept`;
|
||||||
|
- CDN и Gateway L1 cache должны учитывать `Accept`;
|
||||||
|
- response должен выставлять `Cache-Control: public, max-age=31536000, immutable` для versioned assets.
|
||||||
|
|
||||||
|
Для ручного `<picture>`/`srcset` можно добавить явный формат позже:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/images/{assetId}/v{version}/{preset}?w=640&q=80&f=avif
|
||||||
|
/images/{assetId}/v{version}/{preset}?w=640&q=80&f=webp
|
||||||
|
/images/{assetId}/v{version}/{preset}?w=640&q=80&f=jpg
|
||||||
|
```
|
||||||
|
|
||||||
## External imgproxy
|
## External imgproxy
|
||||||
|
|
||||||
|
|||||||
224
docs/backend-contract-draft.md
Normal file
224
docs/backend-contract-draft.md
Normal file
@@ -0,0 +1,224 @@
|
|||||||
|
# Черновик Backend Контракта
|
||||||
|
|
||||||
|
Это черновик будущего бизнес-контракта для NestJS backend. Сейчас реализованы system endpoints, Swagger и placeholder для internal image ensure.
|
||||||
|
|
||||||
|
## System
|
||||||
|
|
||||||
|
```text
|
||||||
|
GET /api/health
|
||||||
|
GET /docs
|
||||||
|
GET /docs-json
|
||||||
|
```
|
||||||
|
|
||||||
|
NestJS backend отдаёт JSON, metadata, statuses и URLs. Public image origin находится в Fastify Gateway. Backend владеет PostgreSQL, S3 orchestration и RabbitMQ jobs.
|
||||||
|
|
||||||
|
## Internal Image Ensure
|
||||||
|
|
||||||
|
Этот internal endpoint вызывается Gateway на L1 miss. Gateway не ходит в DB/S3 напрямую.
|
||||||
|
|
||||||
|
```text
|
||||||
|
POST /api/internal/images/ensure
|
||||||
|
```
|
||||||
|
|
||||||
|
Текущий статус реализации - endpoint зарезервирован и возвращает `501 Not Implemented`. Gateway `/images/*` пока тоже возвращает placeholder `501 Not Implemented`.
|
||||||
|
|
||||||
|
Request body:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"assetId": "asset_123",
|
||||||
|
"version": 4,
|
||||||
|
"preset": "card",
|
||||||
|
"width": 640,
|
||||||
|
"quality": 80,
|
||||||
|
"format": "webp"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Query params:
|
||||||
|
|
||||||
|
| Param | Описание |
|
||||||
|
|---|---|
|
||||||
|
| `w` | целевая ширина, должна быть разрешена preset или округлена до ближайшей разрешённой |
|
||||||
|
| `q` | качество, должно быть из allowlist качества |
|
||||||
|
| `f` | `auto`, `avif`, `webp`, `jpg`; для Next/image по умолчанию `auto` |
|
||||||
|
|
||||||
|
Responsibilities:
|
||||||
|
|
||||||
|
- проверить `assetId` и `preset`;
|
||||||
|
- вычислить deterministic `variantHash`;
|
||||||
|
- проверить PostgreSQL и S3;
|
||||||
|
- если variant готов в S3, вернуть bytes или stream metadata для Gateway;
|
||||||
|
- если variant отсутствует, создать/переиспользовать variant row;
|
||||||
|
- поставить `generate-variant` job в RabbitMQ;
|
||||||
|
- дождаться `ready` до timeout, чтобы первый `next/image` request мог получить картинку;
|
||||||
|
- вернуть image response или metadata для Gateway.
|
||||||
|
|
||||||
|
Response headers:
|
||||||
|
|
||||||
|
```http
|
||||||
|
Content-Type: image/avif | image/webp | image/jpeg
|
||||||
|
Cache-Control: public, max-age=31536000, immutable
|
||||||
|
Vary: Accept
|
||||||
|
ETag: "..."
|
||||||
|
```
|
||||||
|
|
||||||
|
Ошибки:
|
||||||
|
|
||||||
|
| Status | Когда |
|
||||||
|
|---|---|
|
||||||
|
| `400` | некорректные query params |
|
||||||
|
| `404` | asset или preset не найден |
|
||||||
|
| `409` | variant уже генерируется и sync ожидание отключено |
|
||||||
|
| `422` | source image нельзя обработать |
|
||||||
|
| `502` | external imgproxy недоступен |
|
||||||
|
|
||||||
|
Public endpoint реализуется в Fastify Gateway, internal ensure endpoint - в Backend:
|
||||||
|
|
||||||
|
```text
|
||||||
|
client -> CDN -> Gateway /images/* -> Backend ensure -> RabbitMQ -> Worker -> imgproxy -> S3
|
||||||
|
```
|
||||||
|
|
||||||
|
Важно: `/images/*` не должен жить в NestJS `/api`. Gateway остаётся public image origin, Backend остаётся управлением процессов.
|
||||||
|
Public image URL versioned: `/images/{assetId}/v{version}/{preset}?w={width}&q={quality}&f=auto`.
|
||||||
|
|
||||||
|
## Allowed Hosts
|
||||||
|
|
||||||
|
```text
|
||||||
|
GET /allowed-hosts
|
||||||
|
POST /allowed-hosts
|
||||||
|
PATCH /allowed-hosts/:id
|
||||||
|
DELETE /allowed-hosts/:id
|
||||||
|
```
|
||||||
|
|
||||||
|
## Assets
|
||||||
|
|
||||||
|
```text
|
||||||
|
GET /assets
|
||||||
|
POST /assets
|
||||||
|
GET /assets/:id
|
||||||
|
DELETE /assets/:id
|
||||||
|
```
|
||||||
|
|
||||||
|
`POST /assets` request:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"sourceUrl": "https://example.com/photo.jpg"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Responsibilities:
|
||||||
|
|
||||||
|
- validate source URL;
|
||||||
|
- check `allowed_image_hosts`;
|
||||||
|
- create or reuse `image_assets` row;
|
||||||
|
- optionally save original to S3 later.
|
||||||
|
|
||||||
|
## Variants
|
||||||
|
|
||||||
|
```text
|
||||||
|
GET /assets/:id/variants
|
||||||
|
POST /assets/:id/variants
|
||||||
|
POST /variants/:id/regenerate
|
||||||
|
DELETE /variants/:id
|
||||||
|
```
|
||||||
|
|
||||||
|
`POST /assets/:id/variants` request:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"preset": "card",
|
||||||
|
"format": "webp",
|
||||||
|
"width": 640
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Response if ready:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "variant_123",
|
||||||
|
"status": "ready",
|
||||||
|
"url": "http://localhost:8888/images/asset_123/v4/card?w=640&q=80&f=webp"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Response if generation is async:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"id": "variant_123",
|
||||||
|
"status": "pending",
|
||||||
|
"url": null
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Image URLs For UI
|
||||||
|
|
||||||
|
Для ручного UI можно добавить endpoint, который возвращает готовый набор URLs для `<picture>`/`srcset`. Для `next/image` основным контрактом остаётся custom loader из `docs/next-image-provider.md`.
|
||||||
|
|
||||||
|
```text
|
||||||
|
GET /assets/:id/picture?preset=card
|
||||||
|
```
|
||||||
|
|
||||||
|
Example response:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"assetId": "asset_123",
|
||||||
|
"preset": "card",
|
||||||
|
"sources": [
|
||||||
|
{
|
||||||
|
"type": "image/avif",
|
||||||
|
"srcset": "http://localhost:8888/images/asset_123/v4/card?w=320&q=80&f=avif 320w, http://localhost:8888/images/asset_123/v4/card?w=640&q=80&f=avif 640w"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "image/webp",
|
||||||
|
"srcset": "http://localhost:8888/images/asset_123/v4/card?w=320&q=80&f=webp 320w, http://localhost:8888/images/asset_123/v4/card?w=640&q=80&f=webp 640w"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"fallback": {
|
||||||
|
"src": "http://localhost:8888/images/asset_123/v4/card?w=640&q=80&f=jpg",
|
||||||
|
"width": 640,
|
||||||
|
"height": null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Worker Lifecycle
|
||||||
|
|
||||||
|
Worker выполняет задачи из RabbitMQ. Задачи создаёт Backend.
|
||||||
|
|
||||||
|
RabbitMQ topology:
|
||||||
|
|
||||||
|
```text
|
||||||
|
exchange: image-platform.jobs
|
||||||
|
queue: image.generate-variant
|
||||||
|
dead-letter exchange: image-platform.jobs.dlx
|
||||||
|
dead-letter queue: image.generate-variant.dlq
|
||||||
|
```
|
||||||
|
|
||||||
|
Job payload должен быть минимальным:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"jobId": "job_123",
|
||||||
|
"variantId": "variant_123"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Worker читает детали variant из PostgreSQL, вызывает imgproxy, пишет результат в S3 и обновляет status в PostgreSQL.
|
||||||
|
|
||||||
|
Если генерация тяжёлая или не успела завершиться до timeout, Backend может вернуть Gateway `504`, а job продолжит выполняться/retry по очереди.
|
||||||
|
|
||||||
|
PostgreSQL может выступить первой очередью:
|
||||||
|
|
||||||
|
```text
|
||||||
|
SELECT * FROM image_variants
|
||||||
|
WHERE status = 'pending'
|
||||||
|
FOR UPDATE SKIP LOCKED
|
||||||
|
LIMIT 1
|
||||||
|
```
|
||||||
|
|
||||||
|
PostgreSQL-backed очередь не используется как основной механизм: для jobs выбран RabbitMQ.
|
||||||
@@ -1,16 +1,16 @@
|
|||||||
# Черновик Data Model
|
# Data Model
|
||||||
|
|
||||||
Это черновик для будущих миграций PostgreSQL. Реальные таблицы добавим вместе с API.
|
Текущая PostgreSQL модель описана в `packages/database/src/schema.ts` и миграциях Drizzle в `packages/database/drizzle`.
|
||||||
|
|
||||||
## allowed_image_hosts
|
## allowed_image_hosts
|
||||||
|
|
||||||
```text
|
```text
|
||||||
id
|
id uuid pk default gen_random_uuid()
|
||||||
hostname
|
hostname text not null unique
|
||||||
enabled
|
enabled boolean not null default true
|
||||||
description nullable
|
description text nullable
|
||||||
created_at
|
created_at timestamptz not null default now()
|
||||||
updated_at
|
updated_at timestamptz not null default now()
|
||||||
```
|
```
|
||||||
|
|
||||||
Правила normalization:
|
Правила normalization:
|
||||||
@@ -26,53 +26,119 @@ updated_at
|
|||||||
## image_assets
|
## image_assets
|
||||||
|
|
||||||
```text
|
```text
|
||||||
id
|
id uuid pk default gen_random_uuid()
|
||||||
source_url
|
public_id text not null unique
|
||||||
source_host
|
current_version integer not null default 1
|
||||||
source_hash
|
status asset_status not null default active
|
||||||
original_s3_key nullable
|
created_at timestamptz not null default now()
|
||||||
status
|
updated_at timestamptz not null default now()
|
||||||
width nullable
|
|
||||||
height nullable
|
|
||||||
content_type nullable
|
|
||||||
size_bytes nullable
|
|
||||||
created_at
|
|
||||||
updated_at
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
`public_id` - стабильный идентификатор в public URL. `current_version` указывает активную версию source image и используется для cache invalidation без purge.
|
||||||
|
|
||||||
|
## image_asset_versions
|
||||||
|
|
||||||
|
```text
|
||||||
|
id uuid pk default gen_random_uuid()
|
||||||
|
asset_id uuid not null references image_assets(id) on delete cascade
|
||||||
|
version integer not null
|
||||||
|
source_url text not null
|
||||||
|
source_host text not null
|
||||||
|
source_hash text not null
|
||||||
|
original_s3_key text nullable
|
||||||
|
width integer nullable
|
||||||
|
height integer nullable
|
||||||
|
content_type text nullable
|
||||||
|
size_bytes bigint nullable
|
||||||
|
created_at timestamptz not null default now()
|
||||||
|
```
|
||||||
|
|
||||||
|
Каждое изменение source image создаёт новую версию. Старые versioned URLs остаются immutable, новые клиенты получают URL с новым `v{version}`.
|
||||||
|
|
||||||
## image_variants
|
## image_variants
|
||||||
|
|
||||||
```text
|
```text
|
||||||
id
|
id uuid pk default gen_random_uuid()
|
||||||
asset_id
|
asset_id uuid not null references image_assets(id) on delete cascade
|
||||||
preset
|
asset_version_id uuid not null references image_asset_versions(id) on delete cascade
|
||||||
variant_hash
|
asset_version integer not null
|
||||||
format
|
preset text not null
|
||||||
width
|
variant_hash text not null unique
|
||||||
height nullable
|
requested_format requested_format not null default auto
|
||||||
quality
|
format variant_format not null
|
||||||
s3_key
|
width integer not null
|
||||||
status: pending | processing | ready | failed
|
height integer nullable
|
||||||
size_bytes nullable
|
quality integer not null
|
||||||
error nullable
|
s3_key text not null unique
|
||||||
created_at
|
content_type text nullable
|
||||||
updated_at
|
etag text nullable
|
||||||
last_accessed_at nullable
|
status variant_status not null default pending
|
||||||
|
size_bytes bigint nullable
|
||||||
|
error text nullable
|
||||||
|
attempt_count integer not null default 0
|
||||||
|
last_accessed_at timestamptz nullable
|
||||||
|
created_at timestamptz not null default now()
|
||||||
|
updated_at timestamptz not null default now()
|
||||||
|
```
|
||||||
|
|
||||||
|
`requested_format` хранит то, что запросил клиент (`auto`, `avif`, `webp`, `jpg`, `png`). `format` хранит фактический output format после negotiation по `Accept`.
|
||||||
|
|
||||||
|
## Enums
|
||||||
|
|
||||||
|
```text
|
||||||
|
asset_status: active | disabled | deleted
|
||||||
|
requested_format: auto | avif | webp | jpg | png
|
||||||
|
variant_format: avif | webp | jpg | png
|
||||||
|
variant_status: pending | processing | ready | failed
|
||||||
```
|
```
|
||||||
|
|
||||||
## Unique constraints
|
## Unique constraints
|
||||||
|
|
||||||
```text
|
```text
|
||||||
allowed_image_hosts(hostname)
|
allowed_image_hosts(hostname)
|
||||||
image_assets(source_hash)
|
image_assets(public_id)
|
||||||
image_variants(asset_id, variant_hash, format)
|
image_asset_versions(asset_id, version)
|
||||||
|
image_variants(asset_id, asset_version, preset, width, quality, format)
|
||||||
|
image_variants(s3_key)
|
||||||
|
image_variants(variant_hash)
|
||||||
|
```
|
||||||
|
|
||||||
|
Индексы:
|
||||||
|
|
||||||
|
```text
|
||||||
|
image_asset_versions(source_hash)
|
||||||
|
image_variants(status)
|
||||||
```
|
```
|
||||||
|
|
||||||
## S3 layout
|
## S3 layout
|
||||||
|
|
||||||
```text
|
```text
|
||||||
originals/{assetId}/source
|
originals/{assetId}/v{version}/source
|
||||||
variants/{assetId}/{variantHash}.{format}
|
variants/{assetId}/v{version}/{variantHash}.{format}
|
||||||
|
```
|
||||||
|
|
||||||
|
`variantHash` должен включать:
|
||||||
|
|
||||||
|
- `assetId`;
|
||||||
|
- `assetVersion`;
|
||||||
|
- `preset`;
|
||||||
|
- normalized width;
|
||||||
|
- normalized quality;
|
||||||
|
- фактический output format;
|
||||||
|
- параметры transform, влияющие на bytes.
|
||||||
|
|
||||||
|
Для `f=auto` в public URL в S3 всё равно пишется фактический формат:
|
||||||
|
|
||||||
|
```text
|
||||||
|
variants/asset_123/v4/card_w640_q80_avif.avif
|
||||||
|
variants/asset_123/v4/card_w640_q80_webp.webp
|
||||||
|
variants/asset_123/v4/card_w640_q80_jpg.jpg
|
||||||
|
```
|
||||||
|
|
||||||
|
Public URL также versioned:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/images/{assetId}/v{version}/{preset}?w={width}&q={quality}&f=auto
|
||||||
```
|
```
|
||||||
|
|
||||||
## Presets
|
## Presets
|
||||||
|
|||||||
@@ -2,15 +2,18 @@
|
|||||||
|
|
||||||
## Принцип
|
## Принцип
|
||||||
|
|
||||||
В Docker запускаем стабильную инфраструктуру. Кодовые сервисы позже запускаем нодой с hot reload.
|
В Docker запускаем стабильную инфраструктуру. Кодовые сервисы запускаем нодой с hot reload.
|
||||||
|
|
||||||
Сейчас в Docker есть только:
|
Сейчас в Docker есть только:
|
||||||
|
|
||||||
- PostgreSQL;
|
- PostgreSQL;
|
||||||
- MinIO;
|
- MinIO;
|
||||||
- MinIO bucket init.
|
- MinIO bucket init;
|
||||||
|
- imgproxy dev instance;
|
||||||
|
- RabbitMQ.
|
||||||
|
|
||||||
`api`, `worker`, `admin` и `gateway` пока не созданы.
|
`backend` уже создан как NestJS app. `admin` уже создан как чистый Vite React TS app. `gateway` уже создан как Fastify app. `worker` уже создан как RabbitMQ skeleton.
|
||||||
|
Gateway обязателен для Cloudinary-like поведения и интеграции с `next/image`.
|
||||||
|
|
||||||
## Запуск инфраструктуры
|
## Запуск инфраструктуры
|
||||||
|
|
||||||
@@ -45,40 +48,139 @@ pnpm infra:logs
|
|||||||
| PostgreSQL | `localhost:5433` |
|
| PostgreSQL | `localhost:5433` |
|
||||||
| MinIO API | `http://localhost:9000` |
|
| MinIO API | `http://localhost:9000` |
|
||||||
| MinIO Console | `http://localhost:9001` |
|
| MinIO Console | `http://localhost:9001` |
|
||||||
|
| imgproxy | `http://localhost:18080` |
|
||||||
|
| RabbitMQ | `amqp://localhost:5672` |
|
||||||
|
| RabbitMQ Management | `http://localhost:15672` |
|
||||||
|
| Backend API | `http://localhost:3001/api` |
|
||||||
|
| Swagger | `http://localhost:3001/docs` |
|
||||||
|
| Admin | `http://localhost:5173` |
|
||||||
|
| Gateway | `http://localhost:8888` |
|
||||||
|
|
||||||
|
Если `localhost:8888` занят старым `image-gateway`, остановите старый stack или задайте `GATEWAY_PORT=8890` в `.env`.
|
||||||
|
|
||||||
|
## Backend
|
||||||
|
|
||||||
|
Запустить NestJS backend:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pnpm backend:dev
|
||||||
|
```
|
||||||
|
|
||||||
|
Проверки:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:3001/api/health
|
||||||
|
open http://localhost:3001/docs
|
||||||
|
```
|
||||||
|
|
||||||
|
## Database
|
||||||
|
|
||||||
|
Drizzle schema живёт в `packages/database/src/schema.ts`, миграции - в `packages/database/drizzle`.
|
||||||
|
|
||||||
|
Сгенерировать миграцию после изменения schema:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pnpm db:generate
|
||||||
|
```
|
||||||
|
|
||||||
|
Применить миграции к локальному PostgreSQL:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pnpm db:migrate
|
||||||
|
```
|
||||||
|
|
||||||
|
Открыть Drizzle Studio из корня проекта:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pnpm db:studio
|
||||||
|
```
|
||||||
|
|
||||||
|
Проверить database package:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pnpm db:typecheck
|
||||||
|
pnpm db:build
|
||||||
|
```
|
||||||
|
|
||||||
|
## Admin
|
||||||
|
|
||||||
|
Запустить React/Vite admin:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pnpm admin:dev
|
||||||
|
```
|
||||||
|
|
||||||
|
Открыть:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
open http://localhost:5173
|
||||||
|
```
|
||||||
|
|
||||||
|
## Gateway
|
||||||
|
|
||||||
|
Gateway запускается нодой:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pnpm gateway:dev
|
||||||
|
```
|
||||||
|
|
||||||
|
Проверить gateway health:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:8888/health
|
||||||
|
```
|
||||||
|
|
||||||
|
Проверить placeholder image origin route:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -i "http://localhost:8888/images/asset_123/v4/card?w=640&q=80&f=auto"
|
||||||
|
```
|
||||||
|
|
||||||
|
Ожидаемый текущий статус - `501`, потому что S3/imgproxy генерация ещё не реализована.
|
||||||
|
|
||||||
|
## Worker
|
||||||
|
|
||||||
|
Worker запускается нодой и сейчас только объявляет RabbitMQ topology, слушает `image.generate-variant` и отправляет полученные jobs в DLQ, потому что генерация ещё не реализована.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pnpm worker:dev
|
||||||
|
```
|
||||||
|
|
||||||
|
Проверить worker package без запуска consumer:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pnpm worker:typecheck
|
||||||
|
pnpm worker:build
|
||||||
|
```
|
||||||
|
|
||||||
## Будущий dev flow
|
## Будущий dev flow
|
||||||
|
|
||||||
Когда появятся приложения:
|
Текущая и будущая схема:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
React/Vite admin localhost:5173
|
React/Vite admin localhost:5173
|
||||||
-> NestJS API localhost:3001
|
-> NestJS backend localhost:3001
|
||||||
-> PostgreSQL localhost:5433
|
-> PostgreSQL localhost:5433
|
||||||
-> MinIO localhost:9000
|
-> MinIO localhost:9000
|
||||||
|
-> RabbitMQ localhost:5672
|
||||||
|
|
||||||
worker node process
|
worker node process
|
||||||
-> PostgreSQL
|
-> PostgreSQL
|
||||||
-> MinIO
|
-> MinIO
|
||||||
-> external imgproxy
|
-> RabbitMQ
|
||||||
|
-> imgproxy localhost:18080
|
||||||
|
|
||||||
gateway Caddy/Souin localhost:8888
|
Fastify gateway localhost:8888
|
||||||
-> S3/MinIO ready variant
|
-> L1 memory cache
|
||||||
-> API/generator fallback on host.docker.internal:3001
|
-> Backend internal ensure endpoint
|
||||||
```
|
```
|
||||||
|
|
||||||
Для Linux gateway container должен видеть host services через:
|
## imgproxy для разработки
|
||||||
|
|
||||||
```yaml
|
В dev compose поднимается локальный `imgproxy`, опубликованный только на `127.0.0.1:18080`:
|
||||||
extra_hosts:
|
|
||||||
- "host.docker.internal:host-gateway"
|
|
||||||
```
|
|
||||||
|
|
||||||
## External imgproxy для разработки
|
|
||||||
|
|
||||||
`imgproxy` не входит в `image-platform` stack. Для локальной разработки можно использовать любой внешний endpoint и прописать его в `.env`:
|
|
||||||
|
|
||||||
```env
|
```env
|
||||||
IMGPROXY_UPSTREAM=http://localhost:18080
|
IMGPROXY_UPSTREAM=http://localhost:18080
|
||||||
```
|
```
|
||||||
|
|
||||||
Если нужен локальный standalone imgproxy, его можно запустить отдельно вне этого compose stack. Он остаётся внешней зависимостью платформы.
|
Для production `imgproxy` всё равно рассматривается как внешняя зависимость и может жить на отдельном мощном сервере.
|
||||||
|
|||||||
@@ -63,7 +63,7 @@ Final signed URL:
|
|||||||
## Security rules
|
## Security rules
|
||||||
|
|
||||||
- Не отдавать `IMGPROXY_KEY` и `IMGPROXY_SALT` в браузер.
|
- Не отдавать `IMGPROXY_KEY` и `IMGPROXY_SALT` в браузер.
|
||||||
- Source URL валидировать в API/worker.
|
- Source URL валидировать в Backend/worker.
|
||||||
- Разрешать только `http` и `https`.
|
- Разрешать только `http` и `https`.
|
||||||
- Запрещать localhost, private IP, loopback, link-local.
|
- Запрещать localhost, private IP, loopback, link-local.
|
||||||
- Source host должен быть enabled в `allowed_image_hosts`.
|
- Source host должен быть enabled в `allowed_image_hosts`.
|
||||||
|
|||||||
105
docs/next-image-provider.md
Normal file
105
docs/next-image-provider.md
Normal file
@@ -0,0 +1,105 @@
|
|||||||
|
# Next/image Provider
|
||||||
|
|
||||||
|
`image-platform` должен работать как custom image provider для `next/image`.
|
||||||
|
|
||||||
|
## Next.js contract
|
||||||
|
|
||||||
|
Next.js custom loader получает только:
|
||||||
|
|
||||||
|
- `src`;
|
||||||
|
- `width`;
|
||||||
|
- `quality`.
|
||||||
|
|
||||||
|
Поэтому public image URL должен принимать эти параметры напрямую и сам запускать read-through генерацию при miss.
|
||||||
|
|
||||||
|
## Loader config
|
||||||
|
|
||||||
|
В Next.js приложении используется `loaderFile`:
|
||||||
|
|
||||||
|
```js
|
||||||
|
// next.config.js
|
||||||
|
module.exports = {
|
||||||
|
images: {
|
||||||
|
loader: "custom",
|
||||||
|
loaderFile: "./src/image-platform-loader.js",
|
||||||
|
qualities: [60, 75, 80, 90],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Пример loader:
|
||||||
|
|
||||||
|
```js
|
||||||
|
"use client"
|
||||||
|
|
||||||
|
const imageBaseUrl = process.env.NEXT_PUBLIC_IMAGE_PLATFORM_URL
|
||||||
|
|
||||||
|
export default function imagePlatformLoader({ src, width, quality }) {
|
||||||
|
const normalizedSrc = src.startsWith("/") ? src.slice(1) : src
|
||||||
|
const q = quality || 80
|
||||||
|
|
||||||
|
return `${imageBaseUrl}/images/${normalizedSrc}?w=${width}&q=${q}&f=auto`
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Пример использования:
|
||||||
|
|
||||||
|
```tsx
|
||||||
|
import Image from "next/image"
|
||||||
|
|
||||||
|
export function ProductCard() {
|
||||||
|
return <Image src="asset_123/v4/card" width={640} height={420} alt="Product" />
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Public URL
|
||||||
|
|
||||||
|
```text
|
||||||
|
GET /images/{assetId}/v{version}/{preset}?w={width}&q={quality}&f=auto
|
||||||
|
```
|
||||||
|
|
||||||
|
Сейчас route уже зарезервирован в Fastify Gateway, но возвращает placeholder `501`, пока не реализованы PostgreSQL/S3/imgproxy read-through операции.
|
||||||
|
|
||||||
|
Пример:
|
||||||
|
|
||||||
|
```text
|
||||||
|
https://img.example.com/images/asset_123/v4/card?w=640&q=80&f=auto
|
||||||
|
```
|
||||||
|
|
||||||
|
`src` не должен быть source URL. Это должен быть versioned platform identifier, например `asset_123/v4/card`. Source URL хранится в PostgreSQL и не раскрывается в public image URL.
|
||||||
|
|
||||||
|
`v{version}` меняется при обновлении source image. Старые URL можно кэшировать как immutable без purge.
|
||||||
|
|
||||||
|
## Format auto
|
||||||
|
|
||||||
|
`f=auto` выбирает output format по `Accept` header:
|
||||||
|
|
||||||
|
1. `image/avif`, если клиент поддерживает AVIF и preset разрешает AVIF.
|
||||||
|
2. `image/webp`, если клиент поддерживает WebP и preset разрешает WebP.
|
||||||
|
3. `image/jpeg` или original fallback.
|
||||||
|
|
||||||
|
Для auto format обязательны headers:
|
||||||
|
|
||||||
|
```http
|
||||||
|
Vary: Accept
|
||||||
|
Cache-Control: public, max-age=31536000, immutable
|
||||||
|
Content-Type: image/avif | image/webp | image/jpeg
|
||||||
|
```
|
||||||
|
|
||||||
|
CDN и Gateway L1 cache должны учитывать `Vary: Accept`, иначе можно отдать AVIF клиенту без AVIF support.
|
||||||
|
|
||||||
|
## Read-through behavior
|
||||||
|
|
||||||
|
```text
|
||||||
|
client -> CDN -> Fastify gateway -> L1 memory -> Backend -> RabbitMQ -> Worker -> imgproxy -> S3
|
||||||
|
```
|
||||||
|
|
||||||
|
Поведение:
|
||||||
|
|
||||||
|
- CDN HIT: backend не вызывается.
|
||||||
|
- Gateway L1 HIT: backend не вызывается.
|
||||||
|
- Gateway L1 MISS: Gateway вызывает Backend internal ensure endpoint.
|
||||||
|
- S3 HIT: Backend отдаёт bytes Gateway, Gateway кладёт result в L1.
|
||||||
|
- S3 MISS: Backend ставит RabbitMQ job, Worker генерирует variant через external imgproxy, сохраняет в S3, обновляет PostgreSQL, Backend возвращает bytes Gateway.
|
||||||
|
|
||||||
|
Так достигается Cloudinary-like поведение: первый запрос создаёт derived asset, следующие запросы отдаются из cache/storage.
|
||||||
@@ -46,6 +46,33 @@ services:
|
|||||||
mc anonymous set download local/$${S3_BUCKET}"
|
mc anonymous set download local/$${S3_BUCKET}"
|
||||||
restart: "no"
|
restart: "no"
|
||||||
|
|
||||||
|
imgproxy:
|
||||||
|
image: darthsim/imgproxy:latest
|
||||||
|
restart: unless-stopped
|
||||||
|
ports:
|
||||||
|
- "127.0.0.1:${IMGPROXY_PORT:-18080}:8080"
|
||||||
|
environment:
|
||||||
|
GODEBUG: http2client=0
|
||||||
|
IMGPROXY_KEY: ${IMGPROXY_KEY:-}
|
||||||
|
IMGPROXY_SALT: ${IMGPROXY_SALT:-}
|
||||||
|
IMGPROXY_WORKERS: ${IMGPROXY_WORKERS:-2}
|
||||||
|
IMGPROXY_MAX_SRC_RESOLUTION: ${IMGPROXY_MAX_SRC_RESOLUTION:-20}
|
||||||
|
IMGPROXY_USE_ETAG: "true"
|
||||||
|
IMGPROXY_ENABLE_VIDEO_THUMBNAILS: "false"
|
||||||
|
IMGPROXY_DOWNLOAD_TIMEOUT: ${IMGPROXY_DOWNLOAD_TIMEOUT:-30}
|
||||||
|
IMGPROXY_ALLOWED_SOURCES: ${IMGPROXY_ALLOWED_SOURCES:-}
|
||||||
|
|
||||||
|
rabbitmq:
|
||||||
|
image: rabbitmq:4-management-alpine
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
RABBITMQ_DEFAULT_USER: ${RABBITMQ_DEFAULT_USER:-image}
|
||||||
|
RABBITMQ_DEFAULT_PASS: ${RABBITMQ_DEFAULT_PASS:-image-password}
|
||||||
|
RABBITMQ_DEFAULT_VHOST: ${RABBITMQ_DEFAULT_VHOST:-image_platform}
|
||||||
|
ports:
|
||||||
|
- "127.0.0.1:${RABBITMQ_PORT:-5672}:5672"
|
||||||
|
- "127.0.0.1:${RABBITMQ_MANAGEMENT_PORT:-15672}:15672"
|
||||||
|
|
||||||
volumes:
|
volumes:
|
||||||
postgres-data:
|
postgres-data:
|
||||||
minio-data:
|
minio-data:
|
||||||
|
|||||||
27
package.json
27
package.json
@@ -9,10 +9,35 @@
|
|||||||
"pnpm": ">=10.0.0"
|
"pnpm": ">=10.0.0"
|
||||||
},
|
},
|
||||||
"scripts": {
|
"scripts": {
|
||||||
|
"admin:build": "pnpm --filter @image-platform/admin build",
|
||||||
|
"admin:dev": "pnpm --filter @image-platform/admin dev",
|
||||||
|
"admin:preview": "pnpm --filter @image-platform/admin preview",
|
||||||
|
"admin:typecheck": "pnpm --filter @image-platform/admin typecheck",
|
||||||
|
"backend:build": "pnpm --filter @image-platform/backend build",
|
||||||
|
"backend:dev": "pnpm --filter @image-platform/backend dev",
|
||||||
|
"backend:start": "pnpm --filter @image-platform/backend start",
|
||||||
|
"backend:typecheck": "pnpm --filter @image-platform/backend typecheck",
|
||||||
|
"db:build": "pnpm --filter @image-platform/database build",
|
||||||
|
"db:generate": "pnpm --filter @image-platform/database db:generate",
|
||||||
|
"db:migrate": "pnpm --filter @image-platform/database db:migrate",
|
||||||
|
"db:studio": "pnpm --filter @image-platform/database db:studio",
|
||||||
|
"db:typecheck": "pnpm --filter @image-platform/database typecheck",
|
||||||
|
"gateway:build": "pnpm --filter @image-platform/gateway build",
|
||||||
|
"gateway:dev": "pnpm --filter @image-platform/gateway dev",
|
||||||
|
"gateway:start": "pnpm --filter @image-platform/gateway start",
|
||||||
|
"gateway:typecheck": "pnpm --filter @image-platform/gateway typecheck",
|
||||||
|
"queue:build": "pnpm --filter @image-platform/queue build",
|
||||||
|
"queue:typecheck": "pnpm --filter @image-platform/queue typecheck",
|
||||||
|
"storage:build": "pnpm --filter @image-platform/storage build",
|
||||||
|
"storage:typecheck": "pnpm --filter @image-platform/storage typecheck",
|
||||||
|
"worker:build": "pnpm db:build && pnpm queue:build && pnpm storage:build && pnpm --filter @image-platform/worker build",
|
||||||
|
"worker:dev": "pnpm db:build && pnpm queue:build && pnpm storage:build && pnpm --filter @image-platform/worker dev",
|
||||||
|
"worker:start": "pnpm --filter @image-platform/worker start",
|
||||||
|
"worker:typecheck": "pnpm --filter @image-platform/worker typecheck",
|
||||||
"infra:config": "docker compose -f infra/compose.dev.yml config",
|
"infra:config": "docker compose -f infra/compose.dev.yml config",
|
||||||
"infra:up": "docker compose -f infra/compose.dev.yml up -d",
|
"infra:up": "docker compose -f infra/compose.dev.yml up -d",
|
||||||
"infra:down": "docker compose -f infra/compose.dev.yml down",
|
"infra:down": "docker compose -f infra/compose.dev.yml down",
|
||||||
"infra:logs": "docker compose -f infra/compose.dev.yml logs -f",
|
"infra:logs": "docker compose -f infra/compose.dev.yml logs -f",
|
||||||
"check": "pnpm infra:config"
|
"check": "pnpm infra:config && pnpm db:typecheck && pnpm queue:typecheck && pnpm storage:typecheck && pnpm backend:typecheck && pnpm admin:typecheck && pnpm gateway:typecheck && pnpm worker:typecheck"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
12
packages/database/drizzle.config.ts
Normal file
12
packages/database/drizzle.config.ts
Normal file
@@ -0,0 +1,12 @@
|
|||||||
|
import { defineConfig } from "drizzle-kit"
|
||||||
|
|
||||||
|
export default defineConfig({
|
||||||
|
dbCredentials: {
|
||||||
|
url: process.env.DATABASE_URL ?? "postgres://image:image-password@localhost:5433/image_platform",
|
||||||
|
},
|
||||||
|
dialect: "postgresql",
|
||||||
|
out: "./drizzle",
|
||||||
|
schema: "./src/schema.ts",
|
||||||
|
strict: true,
|
||||||
|
verbose: true,
|
||||||
|
})
|
||||||
72
packages/database/drizzle/0000_calm_magdalene.sql
Normal file
72
packages/database/drizzle/0000_calm_magdalene.sql
Normal file
@@ -0,0 +1,72 @@
|
|||||||
|
CREATE TYPE "public"."asset_status" AS ENUM('active', 'disabled', 'deleted');--> statement-breakpoint
|
||||||
|
CREATE TYPE "public"."requested_format" AS ENUM('auto', 'avif', 'webp', 'jpg', 'png');--> statement-breakpoint
|
||||||
|
CREATE TYPE "public"."variant_format" AS ENUM('avif', 'webp', 'jpg', 'png');--> statement-breakpoint
|
||||||
|
CREATE TYPE "public"."variant_status" AS ENUM('pending', 'processing', 'ready', 'failed');--> statement-breakpoint
|
||||||
|
CREATE TABLE "allowed_image_hosts" (
|
||||||
|
"id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL,
|
||||||
|
"hostname" text NOT NULL,
|
||||||
|
"enabled" boolean DEFAULT true NOT NULL,
|
||||||
|
"description" text,
|
||||||
|
"created_at" timestamp with time zone DEFAULT now() NOT NULL,
|
||||||
|
"updated_at" timestamp with time zone DEFAULT now() NOT NULL
|
||||||
|
);
|
||||||
|
--> statement-breakpoint
|
||||||
|
CREATE TABLE "image_asset_versions" (
|
||||||
|
"id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL,
|
||||||
|
"asset_id" uuid NOT NULL,
|
||||||
|
"version" integer NOT NULL,
|
||||||
|
"source_url" text NOT NULL,
|
||||||
|
"source_host" text NOT NULL,
|
||||||
|
"source_hash" text NOT NULL,
|
||||||
|
"original_s3_key" text,
|
||||||
|
"width" integer,
|
||||||
|
"height" integer,
|
||||||
|
"content_type" text,
|
||||||
|
"size_bytes" bigint,
|
||||||
|
"created_at" timestamp with time zone DEFAULT now() NOT NULL
|
||||||
|
);
|
||||||
|
--> statement-breakpoint
|
||||||
|
CREATE TABLE "image_assets" (
|
||||||
|
"id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL,
|
||||||
|
"public_id" text NOT NULL,
|
||||||
|
"current_version" integer DEFAULT 1 NOT NULL,
|
||||||
|
"status" "asset_status" DEFAULT 'active' NOT NULL,
|
||||||
|
"created_at" timestamp with time zone DEFAULT now() NOT NULL,
|
||||||
|
"updated_at" timestamp with time zone DEFAULT now() NOT NULL
|
||||||
|
);
|
||||||
|
--> statement-breakpoint
|
||||||
|
CREATE TABLE "image_variants" (
|
||||||
|
"id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL,
|
||||||
|
"asset_id" uuid NOT NULL,
|
||||||
|
"asset_version_id" uuid NOT NULL,
|
||||||
|
"asset_version" integer NOT NULL,
|
||||||
|
"preset" text NOT NULL,
|
||||||
|
"variant_hash" text NOT NULL,
|
||||||
|
"requested_format" "requested_format" DEFAULT 'auto' NOT NULL,
|
||||||
|
"format" "variant_format" NOT NULL,
|
||||||
|
"width" integer NOT NULL,
|
||||||
|
"height" integer,
|
||||||
|
"quality" integer NOT NULL,
|
||||||
|
"s3_key" text NOT NULL,
|
||||||
|
"content_type" text,
|
||||||
|
"etag" text,
|
||||||
|
"status" "variant_status" DEFAULT 'pending' NOT NULL,
|
||||||
|
"size_bytes" bigint,
|
||||||
|
"error" text,
|
||||||
|
"attempt_count" integer DEFAULT 0 NOT NULL,
|
||||||
|
"last_accessed_at" timestamp with time zone,
|
||||||
|
"created_at" timestamp with time zone DEFAULT now() NOT NULL,
|
||||||
|
"updated_at" timestamp with time zone DEFAULT now() NOT NULL
|
||||||
|
);
|
||||||
|
--> statement-breakpoint
|
||||||
|
ALTER TABLE "image_asset_versions" ADD CONSTRAINT "image_asset_versions_asset_id_image_assets_id_fk" FOREIGN KEY ("asset_id") REFERENCES "public"."image_assets"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
|
||||||
|
ALTER TABLE "image_variants" ADD CONSTRAINT "image_variants_asset_id_image_assets_id_fk" FOREIGN KEY ("asset_id") REFERENCES "public"."image_assets"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
|
||||||
|
ALTER TABLE "image_variants" ADD CONSTRAINT "image_variants_asset_version_id_image_asset_versions_id_fk" FOREIGN KEY ("asset_version_id") REFERENCES "public"."image_asset_versions"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
|
||||||
|
CREATE UNIQUE INDEX "allowed_image_hosts_hostname_idx" ON "allowed_image_hosts" USING btree ("hostname");--> statement-breakpoint
|
||||||
|
CREATE UNIQUE INDEX "image_asset_versions_asset_version_idx" ON "image_asset_versions" USING btree ("asset_id","version");--> statement-breakpoint
|
||||||
|
CREATE INDEX "image_asset_versions_source_hash_idx" ON "image_asset_versions" USING btree ("source_hash");--> statement-breakpoint
|
||||||
|
CREATE UNIQUE INDEX "image_assets_public_id_idx" ON "image_assets" USING btree ("public_id");--> statement-breakpoint
|
||||||
|
CREATE UNIQUE INDEX "image_variants_lookup_idx" ON "image_variants" USING btree ("asset_id","asset_version","preset","width","quality","format");--> statement-breakpoint
|
||||||
|
CREATE UNIQUE INDEX "image_variants_s3_key_idx" ON "image_variants" USING btree ("s3_key");--> statement-breakpoint
|
||||||
|
CREATE UNIQUE INDEX "image_variants_variant_hash_idx" ON "image_variants" USING btree ("variant_hash");--> statement-breakpoint
|
||||||
|
CREATE INDEX "image_variants_status_idx" ON "image_variants" USING btree ("status");
|
||||||
604
packages/database/drizzle/meta/0000_snapshot.json
Normal file
604
packages/database/drizzle/meta/0000_snapshot.json
Normal file
@@ -0,0 +1,604 @@
|
|||||||
|
{
|
||||||
|
"id": "72292622-d326-46fe-8e6a-90096c7e6634",
|
||||||
|
"prevId": "00000000-0000-0000-0000-000000000000",
|
||||||
|
"version": "7",
|
||||||
|
"dialect": "postgresql",
|
||||||
|
"tables": {
|
||||||
|
"public.allowed_image_hosts": {
|
||||||
|
"name": "allowed_image_hosts",
|
||||||
|
"schema": "",
|
||||||
|
"columns": {
|
||||||
|
"id": {
|
||||||
|
"name": "id",
|
||||||
|
"type": "uuid",
|
||||||
|
"primaryKey": true,
|
||||||
|
"notNull": true,
|
||||||
|
"default": "gen_random_uuid()"
|
||||||
|
},
|
||||||
|
"hostname": {
|
||||||
|
"name": "hostname",
|
||||||
|
"type": "text",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true
|
||||||
|
},
|
||||||
|
"enabled": {
|
||||||
|
"name": "enabled",
|
||||||
|
"type": "boolean",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true,
|
||||||
|
"default": true
|
||||||
|
},
|
||||||
|
"description": {
|
||||||
|
"name": "description",
|
||||||
|
"type": "text",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": false
|
||||||
|
},
|
||||||
|
"created_at": {
|
||||||
|
"name": "created_at",
|
||||||
|
"type": "timestamp with time zone",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true,
|
||||||
|
"default": "now()"
|
||||||
|
},
|
||||||
|
"updated_at": {
|
||||||
|
"name": "updated_at",
|
||||||
|
"type": "timestamp with time zone",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true,
|
||||||
|
"default": "now()"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"indexes": {
|
||||||
|
"allowed_image_hosts_hostname_idx": {
|
||||||
|
"name": "allowed_image_hosts_hostname_idx",
|
||||||
|
"columns": [
|
||||||
|
{
|
||||||
|
"expression": "hostname",
|
||||||
|
"isExpression": false,
|
||||||
|
"asc": true,
|
||||||
|
"nulls": "last"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"isUnique": true,
|
||||||
|
"concurrently": false,
|
||||||
|
"method": "btree",
|
||||||
|
"with": {}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"foreignKeys": {},
|
||||||
|
"compositePrimaryKeys": {},
|
||||||
|
"uniqueConstraints": {},
|
||||||
|
"policies": {},
|
||||||
|
"checkConstraints": {},
|
||||||
|
"isRLSEnabled": false
|
||||||
|
},
|
||||||
|
"public.image_asset_versions": {
|
||||||
|
"name": "image_asset_versions",
|
||||||
|
"schema": "",
|
||||||
|
"columns": {
|
||||||
|
"id": {
|
||||||
|
"name": "id",
|
||||||
|
"type": "uuid",
|
||||||
|
"primaryKey": true,
|
||||||
|
"notNull": true,
|
||||||
|
"default": "gen_random_uuid()"
|
||||||
|
},
|
||||||
|
"asset_id": {
|
||||||
|
"name": "asset_id",
|
||||||
|
"type": "uuid",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true
|
||||||
|
},
|
||||||
|
"version": {
|
||||||
|
"name": "version",
|
||||||
|
"type": "integer",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true
|
||||||
|
},
|
||||||
|
"source_url": {
|
||||||
|
"name": "source_url",
|
||||||
|
"type": "text",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true
|
||||||
|
},
|
||||||
|
"source_host": {
|
||||||
|
"name": "source_host",
|
||||||
|
"type": "text",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true
|
||||||
|
},
|
||||||
|
"source_hash": {
|
||||||
|
"name": "source_hash",
|
||||||
|
"type": "text",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true
|
||||||
|
},
|
||||||
|
"original_s3_key": {
|
||||||
|
"name": "original_s3_key",
|
||||||
|
"type": "text",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": false
|
||||||
|
},
|
||||||
|
"width": {
|
||||||
|
"name": "width",
|
||||||
|
"type": "integer",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": false
|
||||||
|
},
|
||||||
|
"height": {
|
||||||
|
"name": "height",
|
||||||
|
"type": "integer",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": false
|
||||||
|
},
|
||||||
|
"content_type": {
|
||||||
|
"name": "content_type",
|
||||||
|
"type": "text",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": false
|
||||||
|
},
|
||||||
|
"size_bytes": {
|
||||||
|
"name": "size_bytes",
|
||||||
|
"type": "bigint",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": false
|
||||||
|
},
|
||||||
|
"created_at": {
|
||||||
|
"name": "created_at",
|
||||||
|
"type": "timestamp with time zone",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true,
|
||||||
|
"default": "now()"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"indexes": {
|
||||||
|
"image_asset_versions_asset_version_idx": {
|
||||||
|
"name": "image_asset_versions_asset_version_idx",
|
||||||
|
"columns": [
|
||||||
|
{
|
||||||
|
"expression": "asset_id",
|
||||||
|
"isExpression": false,
|
||||||
|
"asc": true,
|
||||||
|
"nulls": "last"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"expression": "version",
|
||||||
|
"isExpression": false,
|
||||||
|
"asc": true,
|
||||||
|
"nulls": "last"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"isUnique": true,
|
||||||
|
"concurrently": false,
|
||||||
|
"method": "btree",
|
||||||
|
"with": {}
|
||||||
|
},
|
||||||
|
"image_asset_versions_source_hash_idx": {
|
||||||
|
"name": "image_asset_versions_source_hash_idx",
|
||||||
|
"columns": [
|
||||||
|
{
|
||||||
|
"expression": "source_hash",
|
||||||
|
"isExpression": false,
|
||||||
|
"asc": true,
|
||||||
|
"nulls": "last"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"isUnique": false,
|
||||||
|
"concurrently": false,
|
||||||
|
"method": "btree",
|
||||||
|
"with": {}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"foreignKeys": {
|
||||||
|
"image_asset_versions_asset_id_image_assets_id_fk": {
|
||||||
|
"name": "image_asset_versions_asset_id_image_assets_id_fk",
|
||||||
|
"tableFrom": "image_asset_versions",
|
||||||
|
"tableTo": "image_assets",
|
||||||
|
"columnsFrom": [
|
||||||
|
"asset_id"
|
||||||
|
],
|
||||||
|
"columnsTo": [
|
||||||
|
"id"
|
||||||
|
],
|
||||||
|
"onDelete": "cascade",
|
||||||
|
"onUpdate": "no action"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"compositePrimaryKeys": {},
|
||||||
|
"uniqueConstraints": {},
|
||||||
|
"policies": {},
|
||||||
|
"checkConstraints": {},
|
||||||
|
"isRLSEnabled": false
|
||||||
|
},
|
||||||
|
"public.image_assets": {
|
||||||
|
"name": "image_assets",
|
||||||
|
"schema": "",
|
||||||
|
"columns": {
|
||||||
|
"id": {
|
||||||
|
"name": "id",
|
||||||
|
"type": "uuid",
|
||||||
|
"primaryKey": true,
|
||||||
|
"notNull": true,
|
||||||
|
"default": "gen_random_uuid()"
|
||||||
|
},
|
||||||
|
"public_id": {
|
||||||
|
"name": "public_id",
|
||||||
|
"type": "text",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true
|
||||||
|
},
|
||||||
|
"current_version": {
|
||||||
|
"name": "current_version",
|
||||||
|
"type": "integer",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true,
|
||||||
|
"default": 1
|
||||||
|
},
|
||||||
|
"status": {
|
||||||
|
"name": "status",
|
||||||
|
"type": "asset_status",
|
||||||
|
"typeSchema": "public",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true,
|
||||||
|
"default": "'active'"
|
||||||
|
},
|
||||||
|
"created_at": {
|
||||||
|
"name": "created_at",
|
||||||
|
"type": "timestamp with time zone",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true,
|
||||||
|
"default": "now()"
|
||||||
|
},
|
||||||
|
"updated_at": {
|
||||||
|
"name": "updated_at",
|
||||||
|
"type": "timestamp with time zone",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true,
|
||||||
|
"default": "now()"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"indexes": {
|
||||||
|
"image_assets_public_id_idx": {
|
||||||
|
"name": "image_assets_public_id_idx",
|
||||||
|
"columns": [
|
||||||
|
{
|
||||||
|
"expression": "public_id",
|
||||||
|
"isExpression": false,
|
||||||
|
"asc": true,
|
||||||
|
"nulls": "last"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"isUnique": true,
|
||||||
|
"concurrently": false,
|
||||||
|
"method": "btree",
|
||||||
|
"with": {}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"foreignKeys": {},
|
||||||
|
"compositePrimaryKeys": {},
|
||||||
|
"uniqueConstraints": {},
|
||||||
|
"policies": {},
|
||||||
|
"checkConstraints": {},
|
||||||
|
"isRLSEnabled": false
|
||||||
|
},
|
||||||
|
"public.image_variants": {
|
||||||
|
"name": "image_variants",
|
||||||
|
"schema": "",
|
||||||
|
"columns": {
|
||||||
|
"id": {
|
||||||
|
"name": "id",
|
||||||
|
"type": "uuid",
|
||||||
|
"primaryKey": true,
|
||||||
|
"notNull": true,
|
||||||
|
"default": "gen_random_uuid()"
|
||||||
|
},
|
||||||
|
"asset_id": {
|
||||||
|
"name": "asset_id",
|
||||||
|
"type": "uuid",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true
|
||||||
|
},
|
||||||
|
"asset_version_id": {
|
||||||
|
"name": "asset_version_id",
|
||||||
|
"type": "uuid",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true
|
||||||
|
},
|
||||||
|
"asset_version": {
|
||||||
|
"name": "asset_version",
|
||||||
|
"type": "integer",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true
|
||||||
|
},
|
||||||
|
"preset": {
|
||||||
|
"name": "preset",
|
||||||
|
"type": "text",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true
|
||||||
|
},
|
||||||
|
"variant_hash": {
|
||||||
|
"name": "variant_hash",
|
||||||
|
"type": "text",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true
|
||||||
|
},
|
||||||
|
"requested_format": {
|
||||||
|
"name": "requested_format",
|
||||||
|
"type": "requested_format",
|
||||||
|
"typeSchema": "public",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true,
|
||||||
|
"default": "'auto'"
|
||||||
|
},
|
||||||
|
"format": {
|
||||||
|
"name": "format",
|
||||||
|
"type": "variant_format",
|
||||||
|
"typeSchema": "public",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true
|
||||||
|
},
|
||||||
|
"width": {
|
||||||
|
"name": "width",
|
||||||
|
"type": "integer",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true
|
||||||
|
},
|
||||||
|
"height": {
|
||||||
|
"name": "height",
|
||||||
|
"type": "integer",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": false
|
||||||
|
},
|
||||||
|
"quality": {
|
||||||
|
"name": "quality",
|
||||||
|
"type": "integer",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true
|
||||||
|
},
|
||||||
|
"s3_key": {
|
||||||
|
"name": "s3_key",
|
||||||
|
"type": "text",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true
|
||||||
|
},
|
||||||
|
"content_type": {
|
||||||
|
"name": "content_type",
|
||||||
|
"type": "text",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": false
|
||||||
|
},
|
||||||
|
"etag": {
|
||||||
|
"name": "etag",
|
||||||
|
"type": "text",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": false
|
||||||
|
},
|
||||||
|
"status": {
|
||||||
|
"name": "status",
|
||||||
|
"type": "variant_status",
|
||||||
|
"typeSchema": "public",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true,
|
||||||
|
"default": "'pending'"
|
||||||
|
},
|
||||||
|
"size_bytes": {
|
||||||
|
"name": "size_bytes",
|
||||||
|
"type": "bigint",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": false
|
||||||
|
},
|
||||||
|
"error": {
|
||||||
|
"name": "error",
|
||||||
|
"type": "text",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": false
|
||||||
|
},
|
||||||
|
"attempt_count": {
|
||||||
|
"name": "attempt_count",
|
||||||
|
"type": "integer",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true,
|
||||||
|
"default": 0
|
||||||
|
},
|
||||||
|
"last_accessed_at": {
|
||||||
|
"name": "last_accessed_at",
|
||||||
|
"type": "timestamp with time zone",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": false
|
||||||
|
},
|
||||||
|
"created_at": {
|
||||||
|
"name": "created_at",
|
||||||
|
"type": "timestamp with time zone",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true,
|
||||||
|
"default": "now()"
|
||||||
|
},
|
||||||
|
"updated_at": {
|
||||||
|
"name": "updated_at",
|
||||||
|
"type": "timestamp with time zone",
|
||||||
|
"primaryKey": false,
|
||||||
|
"notNull": true,
|
||||||
|
"default": "now()"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"indexes": {
|
||||||
|
"image_variants_lookup_idx": {
|
||||||
|
"name": "image_variants_lookup_idx",
|
||||||
|
"columns": [
|
||||||
|
{
|
||||||
|
"expression": "asset_id",
|
||||||
|
"isExpression": false,
|
||||||
|
"asc": true,
|
||||||
|
"nulls": "last"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"expression": "asset_version",
|
||||||
|
"isExpression": false,
|
||||||
|
"asc": true,
|
||||||
|
"nulls": "last"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"expression": "preset",
|
||||||
|
"isExpression": false,
|
||||||
|
"asc": true,
|
||||||
|
"nulls": "last"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"expression": "width",
|
||||||
|
"isExpression": false,
|
||||||
|
"asc": true,
|
||||||
|
"nulls": "last"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"expression": "quality",
|
||||||
|
"isExpression": false,
|
||||||
|
"asc": true,
|
||||||
|
"nulls": "last"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"expression": "format",
|
||||||
|
"isExpression": false,
|
||||||
|
"asc": true,
|
||||||
|
"nulls": "last"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"isUnique": true,
|
||||||
|
"concurrently": false,
|
||||||
|
"method": "btree",
|
||||||
|
"with": {}
|
||||||
|
},
|
||||||
|
"image_variants_s3_key_idx": {
|
||||||
|
"name": "image_variants_s3_key_idx",
|
||||||
|
"columns": [
|
||||||
|
{
|
||||||
|
"expression": "s3_key",
|
||||||
|
"isExpression": false,
|
||||||
|
"asc": true,
|
||||||
|
"nulls": "last"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"isUnique": true,
|
||||||
|
"concurrently": false,
|
||||||
|
"method": "btree",
|
||||||
|
"with": {}
|
||||||
|
},
|
||||||
|
"image_variants_variant_hash_idx": {
|
||||||
|
"name": "image_variants_variant_hash_idx",
|
||||||
|
"columns": [
|
||||||
|
{
|
||||||
|
"expression": "variant_hash",
|
||||||
|
"isExpression": false,
|
||||||
|
"asc": true,
|
||||||
|
"nulls": "last"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"isUnique": true,
|
||||||
|
"concurrently": false,
|
||||||
|
"method": "btree",
|
||||||
|
"with": {}
|
||||||
|
},
|
||||||
|
"image_variants_status_idx": {
|
||||||
|
"name": "image_variants_status_idx",
|
||||||
|
"columns": [
|
||||||
|
{
|
||||||
|
"expression": "status",
|
||||||
|
"isExpression": false,
|
||||||
|
"asc": true,
|
||||||
|
"nulls": "last"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"isUnique": false,
|
||||||
|
"concurrently": false,
|
||||||
|
"method": "btree",
|
||||||
|
"with": {}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"foreignKeys": {
|
||||||
|
"image_variants_asset_id_image_assets_id_fk": {
|
||||||
|
"name": "image_variants_asset_id_image_assets_id_fk",
|
||||||
|
"tableFrom": "image_variants",
|
||||||
|
"tableTo": "image_assets",
|
||||||
|
"columnsFrom": [
|
||||||
|
"asset_id"
|
||||||
|
],
|
||||||
|
"columnsTo": [
|
||||||
|
"id"
|
||||||
|
],
|
||||||
|
"onDelete": "cascade",
|
||||||
|
"onUpdate": "no action"
|
||||||
|
},
|
||||||
|
"image_variants_asset_version_id_image_asset_versions_id_fk": {
|
||||||
|
"name": "image_variants_asset_version_id_image_asset_versions_id_fk",
|
||||||
|
"tableFrom": "image_variants",
|
||||||
|
"tableTo": "image_asset_versions",
|
||||||
|
"columnsFrom": [
|
||||||
|
"asset_version_id"
|
||||||
|
],
|
||||||
|
"columnsTo": [
|
||||||
|
"id"
|
||||||
|
],
|
||||||
|
"onDelete": "cascade",
|
||||||
|
"onUpdate": "no action"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"compositePrimaryKeys": {},
|
||||||
|
"uniqueConstraints": {},
|
||||||
|
"policies": {},
|
||||||
|
"checkConstraints": {},
|
||||||
|
"isRLSEnabled": false
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"enums": {
|
||||||
|
"public.asset_status": {
|
||||||
|
"name": "asset_status",
|
||||||
|
"schema": "public",
|
||||||
|
"values": [
|
||||||
|
"active",
|
||||||
|
"disabled",
|
||||||
|
"deleted"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"public.requested_format": {
|
||||||
|
"name": "requested_format",
|
||||||
|
"schema": "public",
|
||||||
|
"values": [
|
||||||
|
"auto",
|
||||||
|
"avif",
|
||||||
|
"webp",
|
||||||
|
"jpg",
|
||||||
|
"png"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"public.variant_format": {
|
||||||
|
"name": "variant_format",
|
||||||
|
"schema": "public",
|
||||||
|
"values": [
|
||||||
|
"avif",
|
||||||
|
"webp",
|
||||||
|
"jpg",
|
||||||
|
"png"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"public.variant_status": {
|
||||||
|
"name": "variant_status",
|
||||||
|
"schema": "public",
|
||||||
|
"values": [
|
||||||
|
"pending",
|
||||||
|
"processing",
|
||||||
|
"ready",
|
||||||
|
"failed"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"schemas": {},
|
||||||
|
"sequences": {},
|
||||||
|
"roles": {},
|
||||||
|
"policies": {},
|
||||||
|
"views": {},
|
||||||
|
"_meta": {
|
||||||
|
"columns": {},
|
||||||
|
"schemas": {},
|
||||||
|
"tables": {}
|
||||||
|
}
|
||||||
|
}
|
||||||
13
packages/database/drizzle/meta/_journal.json
Normal file
13
packages/database/drizzle/meta/_journal.json
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
{
|
||||||
|
"version": "7",
|
||||||
|
"dialect": "postgresql",
|
||||||
|
"entries": [
|
||||||
|
{
|
||||||
|
"idx": 0,
|
||||||
|
"version": "7",
|
||||||
|
"when": 1777963363578,
|
||||||
|
"tag": "0000_calm_magdalene",
|
||||||
|
"breakpoints": true
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
30
packages/database/package.json
Normal file
30
packages/database/package.json
Normal file
@@ -0,0 +1,30 @@
|
|||||||
|
{
|
||||||
|
"name": "@image-platform/database",
|
||||||
|
"version": "0.1.0",
|
||||||
|
"private": true,
|
||||||
|
"type": "module",
|
||||||
|
"exports": {
|
||||||
|
".": {
|
||||||
|
"types": "./src/index.ts",
|
||||||
|
"default": "./dist/index.js"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"types": "./src/index.ts",
|
||||||
|
"scripts": {
|
||||||
|
"build": "tsc -p tsconfig.build.json",
|
||||||
|
"db:generate": "drizzle-kit generate --config drizzle.config.ts",
|
||||||
|
"db:migrate": "drizzle-kit migrate --config drizzle.config.ts",
|
||||||
|
"db:studio": "drizzle-kit studio --config drizzle.config.ts",
|
||||||
|
"typecheck": "tsc --noEmit -p tsconfig.json"
|
||||||
|
},
|
||||||
|
"dependencies": {
|
||||||
|
"drizzle-orm": "^0.45.2",
|
||||||
|
"pg": "^8.20.0"
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"@types/node": "^25.6.0",
|
||||||
|
"@types/pg": "^8.20.0",
|
||||||
|
"drizzle-kit": "^0.31.10",
|
||||||
|
"typescript": "^6.0.3"
|
||||||
|
}
|
||||||
|
}
|
||||||
21
packages/database/src/client.ts
Normal file
21
packages/database/src/client.ts
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
import { drizzle } from "drizzle-orm/node-postgres"
|
||||||
|
import pg from "pg"
|
||||||
|
|
||||||
|
import * as schema from "./schema.js"
|
||||||
|
|
||||||
|
const { Pool } = pg
|
||||||
|
|
||||||
|
export type DatabasePool = pg.Pool
|
||||||
|
export type Database = ReturnType<typeof createDatabase>
|
||||||
|
|
||||||
|
export function createDatabasePool(databaseUrl = process.env.DATABASE_URL) {
|
||||||
|
if (!databaseUrl) {
|
||||||
|
throw new Error("DATABASE_URL is required")
|
||||||
|
}
|
||||||
|
|
||||||
|
return new Pool({ connectionString: databaseUrl })
|
||||||
|
}
|
||||||
|
|
||||||
|
export function createDatabase(pool: DatabasePool) {
|
||||||
|
return drizzle(pool, { schema })
|
||||||
|
}
|
||||||
2
packages/database/src/index.ts
Normal file
2
packages/database/src/index.ts
Normal file
@@ -0,0 +1,2 @@
|
|||||||
|
export { createDatabase, createDatabasePool } from "./client.js"
|
||||||
|
export * from "./schema.js"
|
||||||
113
packages/database/src/schema.ts
Normal file
113
packages/database/src/schema.ts
Normal file
@@ -0,0 +1,113 @@
|
|||||||
|
import {
|
||||||
|
bigint,
|
||||||
|
boolean,
|
||||||
|
index,
|
||||||
|
integer,
|
||||||
|
pgEnum,
|
||||||
|
pgTable,
|
||||||
|
text,
|
||||||
|
timestamp,
|
||||||
|
uniqueIndex,
|
||||||
|
uuid,
|
||||||
|
} from "drizzle-orm/pg-core"
|
||||||
|
|
||||||
|
export const assetStatusEnum = pgEnum("asset_status", ["active", "disabled", "deleted"])
|
||||||
|
export const variantFormatEnum = pgEnum("variant_format", ["avif", "webp", "jpg", "png"])
|
||||||
|
export const requestedFormatEnum = pgEnum("requested_format", ["auto", "avif", "webp", "jpg", "png"])
|
||||||
|
export const variantStatusEnum = pgEnum("variant_status", ["pending", "processing", "ready", "failed"])
|
||||||
|
|
||||||
|
const timestamps = {
|
||||||
|
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
|
||||||
|
updatedAt: timestamp("updated_at", { withTimezone: true }).notNull().defaultNow(),
|
||||||
|
}
|
||||||
|
|
||||||
|
export const allowedImageHosts = pgTable(
|
||||||
|
"allowed_image_hosts",
|
||||||
|
{
|
||||||
|
id: uuid("id").primaryKey().defaultRandom(),
|
||||||
|
hostname: text("hostname").notNull(),
|
||||||
|
enabled: boolean("enabled").notNull().default(true),
|
||||||
|
description: text("description"),
|
||||||
|
...timestamps,
|
||||||
|
},
|
||||||
|
(table) => [uniqueIndex("allowed_image_hosts_hostname_idx").on(table.hostname)],
|
||||||
|
)
|
||||||
|
|
||||||
|
export const imageAssets = pgTable(
|
||||||
|
"image_assets",
|
||||||
|
{
|
||||||
|
id: uuid("id").primaryKey().defaultRandom(),
|
||||||
|
publicId: text("public_id").notNull(),
|
||||||
|
currentVersion: integer("current_version").notNull().default(1),
|
||||||
|
status: assetStatusEnum("status").notNull().default("active"),
|
||||||
|
...timestamps,
|
||||||
|
},
|
||||||
|
(table) => [uniqueIndex("image_assets_public_id_idx").on(table.publicId)],
|
||||||
|
)
|
||||||
|
|
||||||
|
export const imageAssetVersions = pgTable(
|
||||||
|
"image_asset_versions",
|
||||||
|
{
|
||||||
|
id: uuid("id").primaryKey().defaultRandom(),
|
||||||
|
assetId: uuid("asset_id")
|
||||||
|
.notNull()
|
||||||
|
.references(() => imageAssets.id, { onDelete: "cascade" }),
|
||||||
|
version: integer("version").notNull(),
|
||||||
|
sourceUrl: text("source_url").notNull(),
|
||||||
|
sourceHost: text("source_host").notNull(),
|
||||||
|
sourceHash: text("source_hash").notNull(),
|
||||||
|
originalS3Key: text("original_s3_key"),
|
||||||
|
width: integer("width"),
|
||||||
|
height: integer("height"),
|
||||||
|
contentType: text("content_type"),
|
||||||
|
sizeBytes: bigint("size_bytes", { mode: "number" }),
|
||||||
|
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
|
||||||
|
},
|
||||||
|
(table) => [
|
||||||
|
uniqueIndex("image_asset_versions_asset_version_idx").on(table.assetId, table.version),
|
||||||
|
index("image_asset_versions_source_hash_idx").on(table.sourceHash),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
export const imageVariants = pgTable(
|
||||||
|
"image_variants",
|
||||||
|
{
|
||||||
|
id: uuid("id").primaryKey().defaultRandom(),
|
||||||
|
assetId: uuid("asset_id")
|
||||||
|
.notNull()
|
||||||
|
.references(() => imageAssets.id, { onDelete: "cascade" }),
|
||||||
|
assetVersionId: uuid("asset_version_id")
|
||||||
|
.notNull()
|
||||||
|
.references(() => imageAssetVersions.id, { onDelete: "cascade" }),
|
||||||
|
assetVersion: integer("asset_version").notNull(),
|
||||||
|
preset: text("preset").notNull(),
|
||||||
|
variantHash: text("variant_hash").notNull(),
|
||||||
|
requestedFormat: requestedFormatEnum("requested_format").notNull().default("auto"),
|
||||||
|
format: variantFormatEnum("format").notNull(),
|
||||||
|
width: integer("width").notNull(),
|
||||||
|
height: integer("height"),
|
||||||
|
quality: integer("quality").notNull(),
|
||||||
|
s3Key: text("s3_key").notNull(),
|
||||||
|
contentType: text("content_type"),
|
||||||
|
etag: text("etag"),
|
||||||
|
status: variantStatusEnum("status").notNull().default("pending"),
|
||||||
|
sizeBytes: bigint("size_bytes", { mode: "number" }),
|
||||||
|
error: text("error"),
|
||||||
|
attemptCount: integer("attempt_count").notNull().default(0),
|
||||||
|
lastAccessedAt: timestamp("last_accessed_at", { withTimezone: true }),
|
||||||
|
...timestamps,
|
||||||
|
},
|
||||||
|
(table) => [
|
||||||
|
uniqueIndex("image_variants_lookup_idx").on(
|
||||||
|
table.assetId,
|
||||||
|
table.assetVersion,
|
||||||
|
table.preset,
|
||||||
|
table.width,
|
||||||
|
table.quality,
|
||||||
|
table.format,
|
||||||
|
),
|
||||||
|
uniqueIndex("image_variants_s3_key_idx").on(table.s3Key),
|
||||||
|
uniqueIndex("image_variants_variant_hash_idx").on(table.variantHash),
|
||||||
|
index("image_variants_status_idx").on(table.status),
|
||||||
|
],
|
||||||
|
)
|
||||||
7
packages/database/tsconfig.build.json
Normal file
7
packages/database/tsconfig.build.json
Normal file
@@ -0,0 +1,7 @@
|
|||||||
|
{
|
||||||
|
"extends": "./tsconfig.json",
|
||||||
|
"compilerOptions": {
|
||||||
|
"noEmit": false
|
||||||
|
},
|
||||||
|
"exclude": ["dist", "node_modules", "drizzle.config.ts"]
|
||||||
|
}
|
||||||
21
packages/database/tsconfig.json
Normal file
21
packages/database/tsconfig.json
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
{
|
||||||
|
"compilerOptions": {
|
||||||
|
"allowSyntheticDefaultImports": true,
|
||||||
|
"declaration": true,
|
||||||
|
"esModuleInterop": true,
|
||||||
|
"forceConsistentCasingInFileNames": true,
|
||||||
|
"lib": ["ES2023"],
|
||||||
|
"module": "NodeNext",
|
||||||
|
"moduleResolution": "NodeNext",
|
||||||
|
"noUncheckedIndexedAccess": true,
|
||||||
|
"outDir": "./dist",
|
||||||
|
"rootDir": "./src",
|
||||||
|
"skipLibCheck": true,
|
||||||
|
"sourceMap": true,
|
||||||
|
"strict": true,
|
||||||
|
"target": "ES2023",
|
||||||
|
"types": ["node"]
|
||||||
|
},
|
||||||
|
"include": ["src/**/*.ts"],
|
||||||
|
"exclude": ["dist", "node_modules"]
|
||||||
|
}
|
||||||
21
packages/queue/package.json
Normal file
21
packages/queue/package.json
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
{
|
||||||
|
"name": "@image-platform/queue",
|
||||||
|
"version": "0.1.0",
|
||||||
|
"private": true,
|
||||||
|
"type": "module",
|
||||||
|
"exports": {
|
||||||
|
".": {
|
||||||
|
"types": "./src/index.ts",
|
||||||
|
"default": "./dist/index.js"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"types": "./src/index.ts",
|
||||||
|
"scripts": {
|
||||||
|
"build": "tsc -p tsconfig.build.json",
|
||||||
|
"typecheck": "tsc --noEmit -p tsconfig.json"
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"@types/node": "^25.6.0",
|
||||||
|
"typescript": "^6.0.3"
|
||||||
|
}
|
||||||
|
}
|
||||||
2
packages/queue/src/index.ts
Normal file
2
packages/queue/src/index.ts
Normal file
@@ -0,0 +1,2 @@
|
|||||||
|
export * from "./jobs.js"
|
||||||
|
export * from "./topology.js"
|
||||||
37
packages/queue/src/jobs.ts
Normal file
37
packages/queue/src/jobs.ts
Normal file
@@ -0,0 +1,37 @@
|
|||||||
|
export type GenerateVariantJob = {
|
||||||
|
jobId: string
|
||||||
|
variantId: string
|
||||||
|
}
|
||||||
|
|
||||||
|
export function parseGenerateVariantJobBuffer(buffer: Buffer): GenerateVariantJob {
|
||||||
|
const value = JSON.parse(buffer.toString("utf8")) as unknown
|
||||||
|
|
||||||
|
return parseGenerateVariantJob(value)
|
||||||
|
}
|
||||||
|
|
||||||
|
export function parseGenerateVariantJob(value: unknown): GenerateVariantJob {
|
||||||
|
if (!isRecord(value)) {
|
||||||
|
throw new Error("generate variant job must be a JSON object")
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!isNonEmptyString(value.jobId)) {
|
||||||
|
throw new Error("generate variant job must include jobId")
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!isNonEmptyString(value.variantId)) {
|
||||||
|
throw new Error("generate variant job must include variantId")
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
jobId: value.jobId,
|
||||||
|
variantId: value.variantId,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function isRecord(value: unknown): value is Record<string, unknown> {
|
||||||
|
return typeof value === "object" && value !== null && !Array.isArray(value)
|
||||||
|
}
|
||||||
|
|
||||||
|
function isNonEmptyString(value: unknown): value is string {
|
||||||
|
return typeof value === "string" && value.trim().length > 0
|
||||||
|
}
|
||||||
33
packages/queue/src/topology.ts
Normal file
33
packages/queue/src/topology.ts
Normal file
@@ -0,0 +1,33 @@
|
|||||||
|
export type QueueTopology = {
|
||||||
|
generateVariantDeadLetterQueue: string
|
||||||
|
generateVariantDeadLetterRoutingKey: string
|
||||||
|
generateVariantQueue: string
|
||||||
|
generateVariantRoutingKey: string
|
||||||
|
jobsDeadLetterExchange: string
|
||||||
|
jobsExchange: string
|
||||||
|
}
|
||||||
|
|
||||||
|
export const DEFAULT_QUEUE_TOPOLOGY: QueueTopology = {
|
||||||
|
generateVariantDeadLetterQueue: "image.generate-variant.dlq",
|
||||||
|
generateVariantDeadLetterRoutingKey: "image.generate-variant.dlq",
|
||||||
|
generateVariantQueue: "image.generate-variant",
|
||||||
|
generateVariantRoutingKey: "image.generate-variant",
|
||||||
|
jobsDeadLetterExchange: "image-platform.jobs.dlx",
|
||||||
|
jobsExchange: "image-platform.jobs",
|
||||||
|
}
|
||||||
|
|
||||||
|
export function loadQueueTopologyFromEnv(env: NodeJS.ProcessEnv = process.env): QueueTopology {
|
||||||
|
const generateVariantQueue = env.RABBITMQ_GENERATE_VARIANT_QUEUE ?? DEFAULT_QUEUE_TOPOLOGY.generateVariantQueue
|
||||||
|
const generateVariantDeadLetterQueue =
|
||||||
|
env.RABBITMQ_GENERATE_VARIANT_DLQ ?? DEFAULT_QUEUE_TOPOLOGY.generateVariantDeadLetterQueue
|
||||||
|
|
||||||
|
return {
|
||||||
|
generateVariantDeadLetterQueue,
|
||||||
|
generateVariantDeadLetterRoutingKey:
|
||||||
|
env.RABBITMQ_GENERATE_VARIANT_DLQ_ROUTING_KEY ?? generateVariantDeadLetterQueue,
|
||||||
|
generateVariantQueue,
|
||||||
|
generateVariantRoutingKey: env.RABBITMQ_GENERATE_VARIANT_ROUTING_KEY ?? generateVariantQueue,
|
||||||
|
jobsDeadLetterExchange: env.RABBITMQ_GENERATE_VARIANT_DLX ?? DEFAULT_QUEUE_TOPOLOGY.jobsDeadLetterExchange,
|
||||||
|
jobsExchange: env.RABBITMQ_JOBS_EXCHANGE ?? DEFAULT_QUEUE_TOPOLOGY.jobsExchange,
|
||||||
|
}
|
||||||
|
}
|
||||||
7
packages/queue/tsconfig.build.json
Normal file
7
packages/queue/tsconfig.build.json
Normal file
@@ -0,0 +1,7 @@
|
|||||||
|
{
|
||||||
|
"extends": "./tsconfig.json",
|
||||||
|
"compilerOptions": {
|
||||||
|
"noEmit": false
|
||||||
|
},
|
||||||
|
"exclude": ["dist", "node_modules", "**/*.spec.ts"]
|
||||||
|
}
|
||||||
21
packages/queue/tsconfig.json
Normal file
21
packages/queue/tsconfig.json
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
{
|
||||||
|
"compilerOptions": {
|
||||||
|
"allowSyntheticDefaultImports": true,
|
||||||
|
"declaration": true,
|
||||||
|
"esModuleInterop": true,
|
||||||
|
"forceConsistentCasingInFileNames": true,
|
||||||
|
"lib": ["ES2023"],
|
||||||
|
"module": "NodeNext",
|
||||||
|
"moduleResolution": "NodeNext",
|
||||||
|
"noUncheckedIndexedAccess": true,
|
||||||
|
"outDir": "./dist",
|
||||||
|
"rootDir": "./src",
|
||||||
|
"skipLibCheck": true,
|
||||||
|
"sourceMap": true,
|
||||||
|
"strict": true,
|
||||||
|
"target": "ES2023",
|
||||||
|
"types": ["node"]
|
||||||
|
},
|
||||||
|
"include": ["src/**/*.ts"],
|
||||||
|
"exclude": ["dist", "node_modules"]
|
||||||
|
}
|
||||||
24
packages/storage/package.json
Normal file
24
packages/storage/package.json
Normal file
@@ -0,0 +1,24 @@
|
|||||||
|
{
|
||||||
|
"name": "@image-platform/storage",
|
||||||
|
"version": "0.1.0",
|
||||||
|
"private": true,
|
||||||
|
"type": "module",
|
||||||
|
"exports": {
|
||||||
|
".": {
|
||||||
|
"types": "./src/index.ts",
|
||||||
|
"default": "./dist/index.js"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"types": "./src/index.ts",
|
||||||
|
"scripts": {
|
||||||
|
"build": "tsc -p tsconfig.build.json",
|
||||||
|
"typecheck": "tsc --noEmit -p tsconfig.json"
|
||||||
|
},
|
||||||
|
"dependencies": {
|
||||||
|
"@aws-sdk/client-s3": "^3.1042.0"
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"@types/node": "^25.6.0",
|
||||||
|
"typescript": "^6.0.3"
|
||||||
|
}
|
||||||
|
}
|
||||||
20
packages/storage/src/client.ts
Normal file
20
packages/storage/src/client.ts
Normal file
@@ -0,0 +1,20 @@
|
|||||||
|
import { S3Client, type S3ClientConfig } from "@aws-sdk/client-s3"
|
||||||
|
|
||||||
|
import type { StorageConfig } from "./config.js"
|
||||||
|
|
||||||
|
export function createS3Client(config: StorageConfig) {
|
||||||
|
const clientConfig: S3ClientConfig = {
|
||||||
|
endpoint: config.endpoint,
|
||||||
|
forcePathStyle: config.forcePathStyle,
|
||||||
|
region: config.region,
|
||||||
|
}
|
||||||
|
|
||||||
|
if (config.accessKeyId && config.secretAccessKey) {
|
||||||
|
clientConfig.credentials = {
|
||||||
|
accessKeyId: config.accessKeyId,
|
||||||
|
secretAccessKey: config.secretAccessKey,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return new S3Client(clientConfig)
|
||||||
|
}
|
||||||
31
packages/storage/src/config.ts
Normal file
31
packages/storage/src/config.ts
Normal file
@@ -0,0 +1,31 @@
|
|||||||
|
export type StorageConfig = {
|
||||||
|
accessKeyId?: string
|
||||||
|
bucket: string
|
||||||
|
endpoint?: string
|
||||||
|
forcePathStyle: boolean
|
||||||
|
region: string
|
||||||
|
secretAccessKey?: string
|
||||||
|
}
|
||||||
|
|
||||||
|
export function loadStorageConfigFromEnv(env: NodeJS.ProcessEnv = process.env): StorageConfig {
|
||||||
|
return {
|
||||||
|
accessKeyId: normalizeOptionalString(env.S3_ACCESS_KEY_ID),
|
||||||
|
bucket: env.S3_BUCKET ?? "image-platform",
|
||||||
|
endpoint: normalizeOptionalString(env.S3_ENDPOINT),
|
||||||
|
forcePathStyle: parseBoolean(env.S3_FORCE_PATH_STYLE, true),
|
||||||
|
region: env.S3_REGION ?? "us-east-1",
|
||||||
|
secretAccessKey: normalizeOptionalString(env.S3_SECRET_ACCESS_KEY),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function normalizeOptionalString(value: string | undefined) {
|
||||||
|
return value && value.trim().length > 0 ? value : undefined
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseBoolean(value: string | undefined, fallback: boolean) {
|
||||||
|
if (value === undefined) {
|
||||||
|
return fallback
|
||||||
|
}
|
||||||
|
|
||||||
|
return ["1", "true", "yes"].includes(value.toLowerCase())
|
||||||
|
}
|
||||||
3
packages/storage/src/index.ts
Normal file
3
packages/storage/src/index.ts
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
export * from "./client.js"
|
||||||
|
export * from "./config.js"
|
||||||
|
export * from "./keys.js"
|
||||||
40
packages/storage/src/keys.ts
Normal file
40
packages/storage/src/keys.ts
Normal file
@@ -0,0 +1,40 @@
|
|||||||
|
export type VariantFormat = "avif" | "jpg" | "png" | "webp"
|
||||||
|
|
||||||
|
export type OriginalImageKeyInput = {
|
||||||
|
assetId: string
|
||||||
|
version: number
|
||||||
|
}
|
||||||
|
|
||||||
|
export type VariantImageKeyInput = OriginalImageKeyInput & {
|
||||||
|
format: VariantFormat
|
||||||
|
variantHash: string
|
||||||
|
}
|
||||||
|
|
||||||
|
export function buildOriginalImageKey(input: OriginalImageKeyInput) {
|
||||||
|
return `originals/${safeSegment(input.assetId, "assetId")}/v${safeVersion(input.version)}/source`
|
||||||
|
}
|
||||||
|
|
||||||
|
export function buildVariantImageKey(input: VariantImageKeyInput) {
|
||||||
|
return [
|
||||||
|
"variants",
|
||||||
|
safeSegment(input.assetId, "assetId"),
|
||||||
|
`v${safeVersion(input.version)}`,
|
||||||
|
`${safeSegment(input.variantHash, "variantHash")}.${input.format}`,
|
||||||
|
].join("/")
|
||||||
|
}
|
||||||
|
|
||||||
|
function safeSegment(value: string, name: string) {
|
||||||
|
if (value.length === 0 || value.includes("/")) {
|
||||||
|
throw new Error(`${name} must be a non-empty S3 key segment`)
|
||||||
|
}
|
||||||
|
|
||||||
|
return value
|
||||||
|
}
|
||||||
|
|
||||||
|
function safeVersion(value: number) {
|
||||||
|
if (!Number.isSafeInteger(value) || value < 1) {
|
||||||
|
throw new Error("version must be a positive integer")
|
||||||
|
}
|
||||||
|
|
||||||
|
return value
|
||||||
|
}
|
||||||
7
packages/storage/tsconfig.build.json
Normal file
7
packages/storage/tsconfig.build.json
Normal file
@@ -0,0 +1,7 @@
|
|||||||
|
{
|
||||||
|
"extends": "./tsconfig.json",
|
||||||
|
"compilerOptions": {
|
||||||
|
"noEmit": false
|
||||||
|
},
|
||||||
|
"exclude": ["dist", "node_modules", "**/*.spec.ts"]
|
||||||
|
}
|
||||||
21
packages/storage/tsconfig.json
Normal file
21
packages/storage/tsconfig.json
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
{
|
||||||
|
"compilerOptions": {
|
||||||
|
"allowSyntheticDefaultImports": true,
|
||||||
|
"declaration": true,
|
||||||
|
"esModuleInterop": true,
|
||||||
|
"forceConsistentCasingInFileNames": true,
|
||||||
|
"lib": ["ES2023"],
|
||||||
|
"module": "NodeNext",
|
||||||
|
"moduleResolution": "NodeNext",
|
||||||
|
"noUncheckedIndexedAccess": true,
|
||||||
|
"outDir": "./dist",
|
||||||
|
"rootDir": "./src",
|
||||||
|
"skipLibCheck": true,
|
||||||
|
"sourceMap": true,
|
||||||
|
"strict": true,
|
||||||
|
"target": "ES2023",
|
||||||
|
"types": ["node"]
|
||||||
|
},
|
||||||
|
"include": ["src/**/*.ts"],
|
||||||
|
"exclude": ["dist", "node_modules"]
|
||||||
|
}
|
||||||
5979
pnpm-lock.yaml
generated
5979
pnpm-lock.yaml
generated
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user